Tracking a Vehicle Using an Unmanned Aerial Vehicle

ABSTRACT

Tracking a vehicle using an unmanned aerial vehicle is disclosed. One or more first images may be received from a first camera of the first unmanned aerial vehicle located at a first location. A unique identifier of a first vehicle may be determined to be not identifiable in at least one of the one or more first images. The first unmanned aerial vehicle may be then repositioned. One or more second images having the second field of view and showing the first vehicle may then be received. The unique identifier of the first vehicle may then be determined based on at least one of the one or more second images.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation-in-part of U.S. patent applicationSer. No. 14/263,916, filed Apr. 28, 2014, which is a continuation ofU.S. patent application Ser. No. 14/187,300, filed Feb. 23, 2014, andnow U.S. Pat. No. 8,982,213, which is a continuation of U.S. patentapplication Ser. No. 13/686,802, filed Nov. 27, 2012, and now U.S. Pat.No. 8,698,895, which claims the benefit of U.S. Provisional PatentApplication No. 61/680,268, filed on Aug. 6, 2012, the disclosures ofwhich are incorporated by reference in their entirety.

FIELD OF THE INVENTION

This invention relates to using an unmanned aerial vehicle to identifyvehicles and track and control the use of parking spaces by thevehicles.

BACKGROUND

Various municipal and private parking operators have extensive legacyparking management systems for managing the use of, and obtainingrevenue for the user of, destination locations for vehicles, such asparking spaces for wheeled motor vehicles. The substantial rework andexpense that traditionally has been required to upgrade such legacysystems to full automation poses a substantial up-front capitalinvestment that prevents many parking operators from performing suchupgrades.

What is needed is a fully automated and autonomous parking managementsystem which easily integrates with existing parking payment systems,including systems that have already been installed and operational forsome time. Doing so unlocks new revenue opportunities and efficienciesfor the parking facility operators that are previously untapped.

SUMMARY

In one aspect, a method of tracking a vehicle using an unmanned aerialvehicle is disclosed. One or more first images are received from a firstcamera of a first unmanned aerial vehicle located at a first location,where the one or more first images have a first field of view and show afirst vehicle. A first characteristic of the first vehicle is determinedbased on at least one of the one or more first images. It is determinedthat a unique identifier of the first vehicle is not identifiable in atleast one of the one or more first images. In response, the firstunmanned aerial vehicle is caused to be repositioned from the firstlocation to a second location that is different from the first location.One or more second images are received at a second time that is afterthe first time from the first camera of the first unmanned aerialvehicle, where the one or more second images have a second field of viewand show the first vehicle. A second characteristic of the first vehicleis determined based on at least one of the one or more second images.The second characteristic of the first vehicle is compared to the firstcharacteristic of the first vehicle. In response, it is determined thesecond characteristic of the first vehicle is approximately equal to thefirst characteristic of the first vehicle. In response, it is determinedthat the first vehicle is present in the one or more second images. Theunique identifier of the first vehicle is determined based on at leastone of the one or more second images. Finally, the unique identifier isidentified. Other embodiments of this aspect include correspondingcomputer systems, apparatus, and computer programs recorded on one ormore computer storage devices, each configured to perform the actions ofthe methods.

Implementations may include one or more of the following features.Causing repositioning of the first unmanned aerial vehicle from thefirst location to the second location may include determining a dynamiccharacteristic of the first vehicle and determining, based on thedynamic characteristic of the first vehicle, the second location of thefirst unmanned aerial vehicle. The dynamic characteristic of the firstvehicle may be at least one of a lane identifier, a direction, a groundspeed, and an acceleration of the first vehicle. The second location ofthe first unmanned aerial vehicle may be determined by estimating alocation of the first vehicle at the second time based on the dynamiccharacteristic of the first vehicle. A location of the unique identifierof the first vehicle at the second time may be determined based on thedynamic characteristic of the first vehicle.

Causing repositioning of the first unmanned aerial vehicle from thefirst location to the second location may include determining the secondlocation of the first unmanned aerial vehicle and forwardinginstructions to the first unmanned aerial vehicle to reposition to thesecond location. A distance between the first unmanned aerial vehicleand the first vehicle may be the same when the first unmanned aerialvehicle is at the first location and when the first unmanned aerialvehicle is at the second location. The at least one of the one or moresecond images may show a front region of the first vehicle or a rearregion of the first vehicle.

Causing repositioning of the first unmanned aerial vehicle from thefirst location to the second location may include determining a dynamiccharacteristic of the first vehicle and determining that a magnitude ofthe dynamic characteristic exceeds a first threshold. In response, thefirst unmanned aerial vehicle may be caused to be repositioned from thefirst location to the second location. The dynamic characteristic may bea ground speed of the first vehicle. The unique identifier of the firstvehicle may be a license plate number, a one-dimensional bar code, amatrix bar code, a parking permit identification number, a disabledplacard identification number, or a vehicle registration tag associatedwith the first vehicle.

Causing repositioning of the first unmanned aerial vehicle from thefirst location to the second location may also include determining,based on at least one of the one or more first images, that the firstvehicle has committed a parking violation. In response, the firstunmanned aerial vehicle may be caused to reposition from the firstlocation to the second location. Determining that the first vehicle hascommitted the parking violation may include determining, based on atleast one of the one or more first images, that the first vehicle isparked in a restricted parking zone. Determining that the first vehiclehas committed the parking violation may also include determining, basedon at least one of the one or more first images, that the first vehiclehas a missing or expired parking permit.

The one or more first images may show the first vehicle and a secondvehicle, and a second unique identifier of the second vehicle may beidentifiable in at least one of the one or more first images. One ormore third images may be received from a second camera, where the one ormore third images have a third field of view and show the first vehicle.The third field of view may be different from and at least partiallyoverlap with the first field of view. An overlap area that defines theoverlap between the one or more first images and the one or more thirdimages may be determined. A handoff zone within the overlap area mayalso be determined, where tracking of the first vehicle is passed fromthe second camera to the first camera after the first vehicle enters thehandoff zone. The second camera may be located on a second unmannedaerial vehicle that is different from the first unmanned aerial vehicle.

The first unmanned aerial vehicle may be caused to be repositioned fromthe first location to the second location by determining a first path tothe second location from the first location, determining that anobstruction is present along the first path, and determining a secondpath to the second location from the first location that does notoverlap with the obstruction. The first path may be a straight line. Twoor more first images may be from the first camera of the first unmannedaerial vehicle located at the first location, and the firstcharacteristic of the first vehicle can include a first staticcharacteristic of the first vehicle and a first dynamic characteristicof the first vehicle. Two or more second images may be received from thefirst camera of the first unmanned aerial vehicle at the secondlocation. The second characteristic of the first vehicle may include asecond static characteristic of the first vehicle and a second dynamiccharacteristic of the first vehicle. The second static characteristic ofthe first vehicle may be compared to the first static characteristic ofthe first vehicle and the second dynamic characteristic of the firstvehicle may be compared to the first dynamic characteristic of the firstvehicle. It may be determined that the second static characteristic isapproximately equal to the first static characteristic and the seconddynamic characteristic is approximately equal to the first dynamiccharacteristic. Finally, it may be determined that the first vehicle ispresent in the second field of view of the first camera based on theapproximate equivalence. The first static characteristic and the secondstatic characteristic may be characteristics of the first vehicle thatare constant over time, and the first dynamic characteristic and thesecond dynamic characteristic may be characteristics of the firstvehicle that change over time.

The first static characteristic and the second static characteristic maybe each one or more of a vehicle make, a vehicle model, a vehicle year,a vehicle color, a vehicle outline, a vehicle size, and a vehicledimension of the first vehicle. The first dynamic characteristic and thesecond dynamic characteristic may be each one or more of a laneidentifier, a direction, a ground speed, and an acceleration of thefirst vehicle.

The one or more first images may show at least one destination location.It may be determined that the first vehicle is stopped within the atleast one destination location at the first time. One or more thirdimages may be received from the first camera of the first unmannedaerial vehicle and it may be determined that the first vehicle has leftthe at least one destination location at a third time.

A cost for the first vehicle to occupy the at least one destinationlocation from the first time to the third time may be determined and thecost may be charged to an account associated with the first vehicle.Payment information for the cost may be received from a third-partyparking payment system. The third-party parking payment system may be athird-party pay-by-phone parking payment system, a third-partypay-by-space parking payment system, a third-party pay-and-displayparking payment system, a third-party pay-by-plate payment system, or athird-party curbside parking meter payment system.

Payment information for the at least one destination location may bereceived from a third-party parking payment system, where the paymentinformation specifies a period of time for which payment was received.It may be determined that the use of the at least one destinationlocation by the first vehicle from the first time to the third time isoutside of the period of time received from the third-party parkingpayment system. In response, data including a unique identifier of thefirst vehicle, the period of time received from the third-party parkingpayment system, an identifier of the destination location, the firsttime, and the third time is provided.

A description of one or more restrictions for use of the at least onedestination location may be obtained. It may be determined that the useof the at least one destination location by the first vehicle from thefirst time to the third time violates at least one of the one or morerestrictions. The least one of the one or more restrictions violated bythe first vehicle may be indicated. For example, a mailing addressassociated with the first vehicle may be automatically identifiedwithout user interaction. A citation requesting payment of a feeassociated with the determined violation of the one or more restrictionsmay be automatically generated without user interaction. An enforcementofficer may be identified to issue a citation for the first vehiclebased on the location of the at least one destination location.

Another method of tracking a vehicle using an unmanned aerial vehicle isdisclosed. One or more first images are captured from a first camera ofa first unmanned aerial vehicle located at a first location, where theone or more first images have a first field of view and show a firstvehicle. A first characteristic of the first vehicle is determined basedon at least one of the one or more first images. It is determined that aunique identifier of the first vehicle is not identifiable in at leastone of the one or more first images. In response, the first unmannedaerial vehicle is repositioned from the first location to a secondlocation that is different from the first location. One or more secondimages are captured from the first camera of the first unmanned aerialvehicle. A second characteristic of the first vehicle is determinedbased on at least one of the one or more second images. The secondcharacteristic of the first vehicle is compared to the firstcharacteristic of the first vehicle. In response, it is determined thatthe second characteristic of the first vehicle is approximately equal tothe first characteristic of the first vehicle. In response, it isdetermined that the first vehicle is present in the one or more secondimages. The unique identifier of the first vehicle is determined basedon at least one of the one or more second images. Finally, the uniqueidentifier is identified. Other embodiments of this aspect includecorresponding computer systems, apparatus, and computer programsrecorded on one or more computer storage devices, each configured toperform the actions of the methods.

Various benefits obtained by the disclosed subject matter include, butare not limited to: (1) offering hands-off self-enforcement for parkingcitations; (2) offering hands-off self-enforcement for parking citationsthat integrates with existing “in-place” parking payment infrastructure;(3) a smaller number of sensors, and associated reduction ininstallation costs, in comparison to techniques relying on proximitysensors that identify the presence of only one or very few vehicles—afactor which can prove significant for the management of parking spacesin large open areas, such as parking lots; (4) leveraging commercialoff-the-shelf (COTS) technologies, and associated reduction in equipmentcosts, for vehicle detection and parking enforcement; (5) improvementsin software-based vehicle detection techniques can be quickly updatedthroughout the system, particularly in an embodiment where imageprocessing is performed remote from the cameras, and realizingimprovements in vehicle detection without hardware changes throughout aparking management system; (6) the ability to rapidly make changes insystem behavior system-wide, or in connection with a more narrowlydefined subset of parking spaces.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 illustrates an example of an overall system 100 according to thedisclosed subject matter.

FIGS. 2A and 2B illustrate examples of images captured by identificationcameras.

FIGS. 3A and 3B illustrate examples of images captured by destinationcameras.

FIG. 4 illustrates how images captured by cameras with overlappingfields of view may be used to identify a vehicle, track the movement ofthe vehicle to a destination location, and identify use of thedestination location by the vehicle.

FIG. 5 is a block diagram that illustrates a computer system 500 uponwhich aspects of the invention may be implemented,

FIGS. 6A and 6B illustrate aspects of a graphical user interface (GUI)for specifying characteristics of a camera's field of view. FIG. 6Aillustrates a portion of the GUI in which an image from a first camerais displayed. FIG. 6B illustrates a portion of the GUI in which an imagefrom a second camera is displayed.

FIG. 7 illustrates an unmanned aerial vehicle tracking a vehicle.

FIG. 8 illustrates a process for tracking a vehicle with an unmannedaerial vehicle.

DETAILED DESCRIPTION

FIG. 1 illustrates an example of an overall system 100 according to thedisclosed subject matter. Network 110 provides data communicationservices among the elements illustrated in FIG. 1. There are manynetworking technologies, including, but not limited to, Ethernet, 802.11wireless, and cellular data networking technologies, which those skilledin the art can reliably and predictably integrate for the exchange ofdata among the illustrated elements. In an embodiment, separate networksmay be used. For example, cameras 120 and 125 and server 140 maycommunicate via a first network, whereas the other elements communicatevia the public Internet. By use of an independent first network, thoseskilled in the art may realize desired goals for reliability andsecurity.

Identification camera 120 illustrates one of a plurality of such camerasincluded in system 100. For the convenience of discussion, only a singleIdentification camera 120 is illustrated in FIG. 1. Identificationcamera 120 is positioned such that it may capture images of vehicleidentifiers, such as, but not limited to, an automobile license plate ora label affixed to a vehicle that provides a vehicle identifier in aformat such as, but not limited to, a QR code or a bar code. Imagescaptured by Identification camera 120, such as image 121, are used inconjunction with images captured by one or more destination cameras 125in order to identify individual vehicles, such as vehicle 130, andrecord their usage of various destination locations. Examples ofdestination locations for wheeled motor vehicles include areasdesignated as appropriate for parking, although possibly subject tovarious restrictions, such as, but not limited to, parking spots, andareas designated as inappropriate for parking, such as, but not limitedto, bus stops, fire lanes, areas around fire hydrants, sidewalks, areaswithin X feet of an intersection, driveways, islands, medians, bicyclelanes, pedestrian crossings. For example, server system 140 may beconfigured to determine when a vehicle is parked at a bus stop, andrequest or take action upon that determination, such as requesting thevehicle be towed or issue a citation by mail. In an embodiment, otherparking violations may be identified, such as a vehicle not beingproperly parked within a designated parking space, parking more than 1foot from a curb, parking with the left wheels toward the curb (parkedfacing wrong direction), back-in angle parking, and abandoned vehicles(left in a particular location for X or more consecutive days). In anembodiment, an Identification camera may also include a mobile orhandheld camera, including, for example, a camera included in asmartphone.

Destination camera 125 illustrates one of a plurality of such camerasincluded in system 100. For the convenience of discussion, only a singledestination camera 125 is illustrated in FIG. 1. Destination camera 125is positioned such that it may capture images of one or more destinationlocations, such as, but not limited to, automobile parking spots.Typically, destination camera 125 will be mounted at an elevation, suchas, but not limited to, on a lighting or telephone pole, or on thefaçade or top of a building. By mounting destination camera 125 at anelevation, the field of view of images, such as image 126, captured bydestination camera 125 can include, and therefore be used to monitor,multiple destination locations. In an embodiment, a destination cameramay also include a mobile or handheld camera, including, for example, acamera included in a smartphone. In an embodiment, a destination cameramay be included in a parking meter body. In an embodiment, a destinationcamera may be provided by a space satellite imaging camera, potentiallyobviating a need for destination cameras installed throughout amunicipality. In an embodiment, a destination camera may be included inan air vehicle, such as, but not limited to, a blimp or other airship,or an unmanned air vehicle, and may be autonomously operated forextended periods of flight.

Although Identification camera 120 and destination camera 125 may beconfigured to each directly communicate with network 110 via respectivecommunication links 127 and 128. In another embodiment, Identificationcamera 120 may be configured to communicate directly with, and relaydata through, destination camera 125, such that rather thanIdentification camera 120 being configured to communicate directly withnetwork 110 via communication link 127, it communicates with destinationcamera 125 via communication link 129, and destination camera 125communicates with network 110 via communication link 128, such thatdestination camera 125 serves to relay data to and from identificationcamera 120. Communication link 129 may be a wireless networking link.Although FIG. 1 illustrates an example with a communication link 129between the two cameras 120 and 125, this may be extended to data beingsent from or to a given camera by relaying the data through multiplecameras. Additionally, a wireless mesh network may be established amonga plurality of cameras, providing a fault-tolerant and self-configuringwireless communication medium among the cameras. By using wirelesscommunications instead of hardwired links, the number of hardwiredcommunications links in the system as a whole, and expense and laborassociated with installing and maintaining such hardwired links, can bereduced. Also, this may serve as a useful alternative to using, forexample, cellular data communications for conveying data to network 110,as typically the cost of cellular data communications is significantlygreater than via a hardwired link. However, wireless communication isnot required, and some or all cameras may communicate via a hardwiredlink.

Server system 140 comprises one or more computer systems which providecentral data storage, data retrieval, and processing services. Thecomputer systems included in server system 140 will generally eachinclude a random access memory 142 and a processor 143. Server system140 includes database 141, which is used to record information such asusage of destination locations, such as parking spaces, by vehicles;vehicle account information; identifications of vehicles obtained viaidentification camera images; observations of vehicles obtained viadestination camera images; destination reservation information; billinginformation; and payment information. Database 141 may be implemented,for example, using software programs such as MySQL, Oracle, or DBASE.Database 141 may comprise multiple database instances executing acrossmultiple computer systems.

In an embodiment, server system 140 is configured to perform vehicle andvehicle identification recognition from images captured byidentification cameras, such as identification camera 120, anddestination cameras, such as destination camera 125. Further server 140is configured to perform decision making based on the recognizedinformation, such as determining the usage of destination locations, andwhen the usage of a destination location is in violation of userestrictions for the destination location. This embodiment, in whichimage processing and decision making are centralized in server system140, exploits Moore's Law, in which the number of transistors in amicroprocessor, and a corresponding amount of data processingcapability, doubles every 18 months, and Nielsen's Law, whichhypothesizes that bandwidth speeds grow by 50% per year. Significantbenefits can be realized by this centralized approach. Not only can theintelligence/feature-richness of server applications be quickly scaledup with ever cheaper and faster CPU power, new features and applicationscan also be quickly rolled out to onsite “thin-client” devices.Manufacturing costs are generally reduced by use of COTS (commercial offthe shelf) components throughout the system. In addition, a highlynetworked system will be well positioned to take advantage of updates intechnology services available via network. Finally, as microprocessorand bandwidth costs continue to decrease, the costs of expansion in sizeor capability can be realized at ever reducing costs.

In another embodiment, some functions may be distributed to othercomponents of the system. For example, identification camera 120 may beconfigured to identify the presence of vehicle identificationinformation, such as license plate numbers, in captured images. In suchan embodiment, the amount of data communication bandwidth betweenidentification camera 120 and server system 140, and the amount ofprocessing which must be performed by server system 140 may besignificantly reduced, with a tradeoff in that identification camera 120needs to be a more sophisticated device capable of performing imageprocessing functions capable of, for example, identifying the presenceof vehicles in images, identifying the presence of vehicleidentification information in images, and/or extracting vehicleidentification information from images. By way of another example, theidentification and destination cameras can be programmed to determinecharacteristics of vehicle images, as detailed below in connection withFIG. 4, rather than perform such determinations using server system 140.How functions may be distributed throughout the system as a whole woulddepend on the anticipated costs of providing distributed functionality(such as providing the necessary processing hardware and programming toidentification and/or destination cameras), weighed against theanticipated costs to convey images captured by the cameras to a centrallocation for processing. Moving programmed functions from one computersystem to another to accommodate communication bottlenecks, such as fromserver system 140 to identification camera 120, and ensuring thatadequate processing resources are provided for various computer systemsto perform these functions is a routine exercise which is well withinthe skill in the art.

Mobile device 150 comprises a programmable computer, an identificationcamera, and wireless data communication capability for communicatingwith server system 140. In an example, if vehicle 130 were determined tobe using a destination location, but a unique identification for vehicle130 had not been determined, mobile device 150 could be dispatched tothe destination location, and an image 151 of vehicle 130 captured, fromwhich a unique identification for vehicle 130 could be obtained. Inadditional to capturing an image of a license plate, as done byidentification camera 120, mobile device 150 can also be configured tocapture a vehicle VIN identifier tag, which may be helpful for vehicleslacking license plates. This helps overcome occasional situations, suchas occlusion of vehicle 130 by another vehicle as it passes through thefield of view for identification camera 120, in which identificationcamera 120 is unable to capture images effective for uniquelyidentifying vehicle 130. Multiple mobile devices may be included in thesystem in order to better ensure timely capture of images across all ofthe destination locations managed by server system 140. In anembodiment, mobile device 150 may also be configured to provide featuresuseful for parking enforcement. For example, mobile device 150 may beconfigured to print out a paper citation for parking violationsidentified by server system 140.

Onsite payment system 160 is one of a plurality of devices which offersa direct physical interface, generally located near one or moredestination locations, for an end user to submit a payment or anagreement to pay for the use of a destination location. Examples ofpayment system 160 include, but are not limited to, a parking meterdedicated to an individual destination location, a payment stationassociated with several street parking spaces, and a payment station fordestination locations provided in a parking lot or garage. In anembodiment, onsite payment system 160 may include a destination camerawhich captures images for one or more destination locations.

End user system 170 is a programmable computer system by which an enduser, such as a driver of vehicle 130, may interact with server system140. Examples of such systems include, but are not limited to, a desktopcomputer system with a web browser application, a smartphone with a webbrowser application or dedicated parking application, and an in-vehiclesystem included in vehicle 130. End user system 170 may be used, forexample, to create an end user account for repeated interactions withserver system 140, reserve use of destination locations, requestidentification of an available destination location, and submit paymentinformation for the use of a destination location.

In an embodiment, end user system 170 may be configured to request andobtain information from server system 140 describing availabledestination locations for a given area. In an example, end user system170 may further be configured to obtain and/or request particularcriteria for acceptable destination locations, such as destinationlocations which are in a parking garage or not on-street parking (suchas a parking lot). In addition to location information, otherinformation such as, but not limited to, cost of use for a givendestination location may be obtained by end user system 170. End usersystem 170 may be further configured to request and obtain informationfor currently available destination locations and/or destinationlocations available at a specified point or range of time in the future.End user system may be further configured to provide a graphical userinterface to the end user showing specific locations, such as on a map,of one or more of the available destination locations obtained fromserver system 140. In an embodiment, where multiple destinationlocations are available at a parking facility (such as a parking garageor parking lot), end user system 170 may be configured to broadlyindicate the availability, possibly indicating a number of availabledestination locations) of a destination location at the parkingfacility, rather than individually identifying each destinationlocation. End user system 170 may be configured to receive an indicationof a selected destination location from the end user. End user system170 may be further configured to provide guidance, such as turn-by-turndirections, to the selected destination location. End user system 170may be further configured to instruct server system 140 to reserve useof an available destination location.

Manual review system 180 is a programmed computer system by which ahuman operator 181 provides assistance to server system 140 foridentifying vehicles, such as vehicle 130. For example, server system140 may determine that an error has occurred in performing an automatedmachine-based determination of a unique identifier for vehicle 130 fromimages, such as image 121, captured by identification camera 120. Inresponse to this error, server system 140 may request assistance fromhuman operator 181 to determine a unique identifier, such as a licenseplate number, for vehicle 130. Manual review system 180 provides a userinterface by which human operator 181 may review image 121, and perhapsother images captured by identification camera 120 or other cameras. Forexample, human operator 181 can rewind the video footage manually andfreeze-frame forward in order to manually determine the vehicle licenseplate number and state/province information. Via this interface, humanoperator 181 may provide an identification of vehicle 130 or indicatethat human operator 181 was unable to determine an identification forvehicle 130, which may result in mobile device 150 being dispatched to adestination location in order to make a positive identification ofvehicle 130. More than one manual review system 180 may be madeavailable for use by server system 140, depending on the volume ofinformation processed by server system 140. Additionally, human operator181 may be a member of a Live Operator team, discussed in more detailbelow.

In an embodiment, one or more of the identification and destinationcameras capture images at a higher frame rate and/or resolution than theimages transmitted from the cameras to server system 140. Although notall of the image data captured by such cameras is transmitted to serversystem 140, in the interest in conserving possibly expensive networkcommunication bandwidth (for example, a cellular data network) andlimited image processing resources available at server system 410, suchcameras can buffer a limited amount of the full image data that theycapture, such as in a circular buffer. Additionally, such cameras areconfigured to respond to requests for such additional image data,including, for example, portions of higher resolution images which maybe useful for license plate recognition. By this mechanism, serversystem 140 may automatically determine that additional images may beuseful for identifying and tracking vehicles, or manual review system180 may request various images in order to resolve an error raised byserver system 140.

Parking operation system 190 is a programmed computer system by which aparking facility operator may interact with and control server system140. Server system 140 is expected to operate autonomously with littleinput required by a parking operator the majority of the time. However,occasionally there are instances when the parking facility operatorpersonnel take an active part in its operation. For example, parkingoperation system 190 may provide a live indication as to how many andwhich destination locations are being used and/or are available. Asanother example, parking operation system 190 may provide a liveindication as to destination locations in which vehicles are violatingusage restrictions, allowing parking facility operator personnel to takeaction for such violations as deemed necessary.

At least one unmanned aerial vehicle 700 may be wirelessly connected tothe network 110. The unmanned aerial vehicle 700 may include a camerathat can be used in place of or in conjunction with one or both of theidentification camera 120 and destination camera 125. Images captured bythe unmanned aerial vehicle 700 may be forwarded to server system 140and subsequently processed by the server system 140. The methods andsystems described below with respect to the mobile device 150, theonsite payment system 160, the end user system 170, the manual reviewsystem 180, the parking operation system 190, and other aspects of thepresent disclosure may be implemented using or in conjunction with theunmanned aerial vehicle 700.

FIG. 2A illustrates an example of an image 121 captured byidentification camera 120, where identification camera 120 is located soas to effectively capture identification information for vehiclespassing by identification camera 120. In an embodiment, identificationcamera 120 streams video containing images of identificationinformation, such as license plate information, to server system 140.Since typical streetlights are approximately 18 feet in height, theyprovide an excellent mounting location for identification cameras whichbenefit from a clear line of sight away from trees and other obstacles,including closely traveling vehicles. Also, as a streetlight is alreadysupplied with power, the identification camera may be installed so as todraw power from the streetlight. In addition, as most city regulationsrequire at least one streetlight per block on each site of the street,streetlight-mounted identification cameras can provide sufficientcoverage for any vehicles traversing the street.

Identification images captured by identification camera 120 are used fortwo primary purposes: (1) obtaining a unique identification of avehicle, such as by reading an identifier from a license plate mountedon the front or rear of the vehicle; and (2) determining variouscharacteristics of the vehicle, so that it may be determined that animage of a vehicle obtained by another camera corresponds to anidentified vehicle, as discussed below in connection with FIG. 4. In theexample identification image shown in FIG. 2A, a view of a vehicle andits license are available for processing by server system 140. Multipleidentification images may be used for determining vehiclecharacteristics such as speed and direction of travel.

In the field of license plate readers, various techniques are known inthe art for reliably capturing images effective for reliable reading ofvehicle identifiers under a variety of illumination and vehicleconditions. One nonlimiting example is described in U.S. Pat. No.7,016,518, which is incorporated by reference in its entirety. In anembodiment, issues arising from the lack of natural illumination duringnight may be avoided where restrictions for the use of destinationlocations are only in effect during daylight hours.

In an embodiment, machine vision techniques for obtaining vehicleidentifiers may be performed by identification camera 120. In such anembodiment, the amount of data sent from identification camera 120 toserver system 140 may be significantly reduced, as a vehicle identifiercan be sent in place of identification images. However, one or moreidentification images may still be transmitted to server system 140 forstorage and/or determination of vehicle characteristics by server system140 (although in an embodiment, this may also be performed byidentification camera 120).

In an embodiment, server system 140 may recognize it is only able toobtain a portion of a vehicle identifier, perhaps as a result of certainparticular conditions, such as the angle of the vehicle or a closelyfollowing vehicle. In such a case, server system may further rely on asecond identification image captured by a second identification camerato obtain the remaining portion of the identifier.

FIG. 2B illustrates an example of an image 121 captured byidentification camera 120, where identification camera 120 is located soas to effectively capture identification information for vehiclesentering and exiting an urban intersection via the roadway extendingalong the axis of view of identification camera 120. In thisidentification image, views of three vehicles are effective forprocessing by server system 140, such that unique identifiers may beobtained for all three vehicles from the single image.

In an embodiment, identification camera 120 may have pan, tilt, and/orzoom capability, which may be used for server system 140 to obtain amore detailed view of a vehicle, or may be used by parking facilityoperator personnel via a user interface on parking operation system 190to monitor particular areas of interest viewable by identificationcamera 120. In such an embodiment, server system 140 may be configuredto perform a coordinate transformation in order to accurately determinevehicle characteristics while varying the field of view ofidentification camera 120.

In an embodiment, when a vehicle appears in the field of view ofidentification camera 120, server system 140 performs one or more of sixtasks. In the first task, server system 140 detects the presence of oneor more vehicles based on one or more images captured by identificationcamera 120. In an embodiment, server system 140 may be configured todetect the presence of a vehicle based on differences in color betweenthe vehicle and the background, and a determination that a size of atotal shape for a potential vehicle is larger than a specified minimalvehicle size adjusted for any applicable zoom level. Specifically, suchan algorithm may compare colors of all the pixels in an image againsteach other. Pixels are divided into rows and columns (for example, anabsolute location 21, 34 may represent a pixel located on Row 21 andColumn 34). Since the image likely includes other non-vehicle objectssuch as, but not limited to, a fire hydrant, pedestrian, cars, grass, orroad marking/paint, the algorithm may be configured to detect groups ofpixels with similar colors (a range of colors that be considered to besimilar may be specified in a system configuration, such as that similarcolors may be a maximum of 2 shades difference, with reference to shadesindicated by a color chart). The locations of these pixels are thenmathematically computed to determine whether they form a shape largerthan the minimal vehicle size (for example, a relative size of 10×30 mayrepresent a shape measuring 10 rows by 30 columns). Vehicles appearingin images captured by identification camera 120 tend to appear to beparallelograms rather than rectangular in shape due to the viewingangle. Therefore, the determination may take this factor into account.

A further refinement could be made to accommodate “holes” orimperfections in a detected shape due to, for example, a reflection ofthe sun, a windshield area, or a roof rack. For example, the algorithmmay specify a number of shapes of a particular minimum size up tocertain distance (or pixels) away from each other, which may correspondto portions of a vehicle such as a hood, roof, and trunk, which aretypically of the same color.

Once a vehicle has been detected, an entry is recorded in a portion ofdatabase 141 corresponding to identification camera 120, along withinformation including, for example, a date, time, color of vehicle, sizeof vehicle, locations of pixels, and zoom level. This information may beused in conjunction with other detections of the vehicle to identity ofthe vehicle. Alternatively, such information may be temporarily recordedin volatile memory, such as RAM.

In the second task, server system 140 detects the presence of licenseplates and/or additional identification, such as, but not limited to,decals, on vehicles in images captured by identification camera 120.Once a license plate and/or additional identification has been detected,an entry is recorded in a portion of database 141 corresponding toidentification camera 120, along with information including, forexample, a date, time, license plate #, state/province, decal details(for example, a type, an identifier, and expiration), locations ofpixels, and zoom level. This information may be used in conjunction withother detections of the vehicle to identify the vehicle in other images,such as images captured by a destination camera.

In the third task, server system 140 identifies vehicles that are movingversus vehicles which are stationary, based on multiple images capturedby identification camera 120.

In the fourth task, server system 140 identifies license plates and/oradditional identifications that are moving versus license plates and/oradditional identifications which are stationary, based on multipleimages captured by identification camera 120. In an example, serversystem 140 may be configured to: (1) based on license plates and/oradditional identification information obtained, such as via OCR, fromthe current image, find corresponding matches in previous images (forexample, by reviewing data stored in database 141 or memory); and (2)once such matches are found, compare the locations of pixels between thecurrent image and previous images, taking into account any change inzoom level. Based on such processing, one or more of the followingscenarios may be detected. (A) if locations of pixels are unchanged orhave unchanged below a predetermined threshold, all license platesand/or additional identifications are deemed to have remainedstationary. In this scenario, a simple update of the date and time isperformed for existing entries in database 141 or memory for thevehicles. (B) if pixels associated with one or more license platesand/or additional identifications are at new locations, the one or morelicense plates and/or additional identifications are determined to havemoved. Server system 140 may calculate speed and direction of travel forthese vehicles and update database 141 or memory with this information,along with updated locations. (C) if one or more license plates and/oradditional identifications have disappeared from the current image,server system 140 may be configured to remove data recorded in database141 or memory for such vehicles. (D) if one or more license platesand/or additional identifications have newly appeared in the currentimage, data associated with the new license plates and/or additionalidentifications may be recorded in database 141 or memory. In anembodiment, server system 140 may determine that one of the new vehiclescorresponds to a vehicle recently determined to have disappeared, andmay treat the view of the vehicle as having been temporarily obstructed.(E) if all license plates and/or additional identifications havedisappeared from the current image, data from database 141 or memoryfrom the preceding interval may be removed in connection with thelicense plates and/or additional identifications therein. However, somedata may be retained in order to perform vehicle tracking to or from anidentification camera, as discussed previously.

In the fifth task, server system 140 mates identified license platesand/or additional identifications to vehicles detected in the firsttask. For example, server system 140 may be configured to compareattributes of license plates and/or additional identifications (such as,for example, absolute pixel location, direction of travel, and speed) tosuch attributes determined for possible vehicles and attempt to mateeach license plate and/or additional identification to a vehicle.

In the first through fifth tasks, server system 140 may apply machinevision techniques to identify additional identification on vehicles forthe purpose of confirming that either the particular vehicle iseligibility of parking in a particular parking space, or the vehiclelicense registration is current and valid. These other types of uniqueidentification include, but are not limited to: a handicap parking decalor mirror-hanger, a resident-only parking decal, and a registrationdecal typically adhered to license plates.

In the first through fifth tasks, OCR (optical character recognition)may be performed on any license plate image(s) in order to obtain analphanumeric license plate number along with state/province of issue.Note the license plate observed can be either the front or the rearlicense plate (typically, identification cameras are provided whichpoint both ways up and down a street). The obtained license plateinformation is used to obtain a unique vehicle identifier used forrecording information about the vehicle in database 141. In certainconfigurations, the first task may be performed locally on theidentification camera where the license plate number and state/provincedata are transmitted to the server system 140 in a concise data format.This is especially true when the underlying communications networkbetween the identification camera 120 and server system 140 does notprovide sufficient bandwidth (for example, 1 Mb/sec may be deemed aminimum threshold for streaming images) or is deemed to be too costly(for example, more than $10/MB). Multiple vehicles may be presentsimultaneously in identification image 121. In an embodiment, serversystem 140 can track up to 10 sets of vehicle license number plateinformation simultaneously from a single identification camera at anygiven time.

In the sixth task a vehicle is “tracked” as it moves across the field ofview of identification camera 120. There is overlap between an imagecaptured by identification camera 120 and a neighboring destinationcamera 125 such that a vehicle is within the field of vision of bothcameras for a minimum of 2 seconds at a speed of 15 MPH (approximately44 feet). In general, neighboring or nearby camera positions aredetermined to ensure this overlap occurs. As the two cameras haveuniquely different angles of view, obtaining overlapped images of anygiven vehicle from both of these cameras is treated as a positiveidentification of the vehicle (via license plate number obtained fromthe identification image 121) along with an exact destination locationthe vehicle has occupied (in the event the vehicle parks).

In an embodiment, preliminary motion detection may be used to reduce thenumber of identification images which are captured and/or analyzed, asthere may be significant periods of time during which no vehicles passthrough the field of view of identification camera 120.

An OCR rate is only one of the useful measurements in assessingidentification performance. For example, while it is impressive that of100 license plates recorded/captured, server system 140 failed to OCRonly 1 of the license plates (in other words, was not able to produce analpha-numeric plate number), if such a configuration (including factorssuch as server system programming, identification camera placement, andidentification camera image quality) failed to capture/register 100other license plates (in other words, additional license plate imageswere not identified as such by server system 140), the overalleffectiveness is only 49.5%. Furthermore, license plate recognitionsystems are typically tuned to read license plates from a particularstate/province along with ones from a number of the surroundingstates/provinces. License plates from various states/provinces havedifferent colors and contrast for letters and background. In anembodiment, the capture rate is at least 95% and the OCR accuracy is atleast 95%, yielding a minimum overall effectiveness of at least 90.25%.

FIG. 3A illustrates an example of an image 126 captured by destinationcamera 125, where destination camera 125 is located on a façade or topof a building looking down onto a portion of an urban street, such thata top view of vehicles is seen in image 126. In association with thefield of view corresponding to image 126 shown in FIG. 3A, nineappropriate destination locations 310 a to 310 i (such as parking spotswith a hourly rate for use) have been specified to server system 140. Ofthese, all but destination location 310 d appear to be in use byvehicles 310 a to 310 i. Also specified to server system 140 isinappropriate destination location 340 (such as a commercial entrance).If a vehicle is determined to have made use of destination location 340by remaining stationary for a period of time, the vehicle will beconsidered in violation, resulting in server system 140 initiatingactions such as, but not limited to, assessing a fine for the violation,or alerting a parking facility operator of the violation. In someembodiments, a vehicle remaining stationary in any area not identifiedas a destination location may be considered a violation. Also seen inimage 126 is vehicle 330, in a traffic lane for vehicles traveling fromright to left in the image. Taken alone, the single image does notindicate whether vehicle 330 is moving or stationary. For example, in apreceding image, vehicle 330 might have been making use of destinationlocation 310 d, and this image may cause server system 140 to determinethat vehicle 330 has just completed its use of destination location 310d.

Destination images captured by destination camera 125 are used for twoprimary purposes: (1) identifying when destination locations are beingor not being used by vehicles; and (2) determining variouscharacteristics of vehicles, so that vehicles may be tracked as theytraverse from camera to camera, by determining whether thecharacteristics determined from images captured by neighboring camerascorrespond to one another, as discussed below in connection with FIG. 4.

FIG. 3B illustrates an example of an image 126 captured by destinationcamera 125, where destination camera 125 is positioned such that a sideview of vehicles is seen in image 126. Such a view may be obtained wheredestination camera 125 is located in onsite parking system 160, such asa parking meter, or on a building or structure opposite from one or moredestination locations of interest. In association with the field of viewcorresponding to image 126 shown in FIG. 3B, four appropriatedestination locations 350 a to 350 d (such as parking spots with ahourly rate for use) have been specified to server system 140. Of these,all but destination location 350 c appear to be in use by vehicles 360 ato 360 d.

In another example, not illustrated in the drawings, destination camera125 may be positioned such that image 126 provides a perspective view ofone or more destination locations. Although FIGS. 3A and 3B illustratethe specification of rectangular areas 310 and 350 for destinationlocations, other shapes may be used, such as polygons (which is usefulwhere destination camera 125 is positioned such that a destinationlocation is seen in perspective). The GUI described below in connectionwith FIGS. 6A and 6B may also be configured to provide user interfaceelements which may be used to specify, with an overlay on an image feed,the position and extent of destination locations within the field ofview of a camera. Additionally, the GUI may provide tools for assigningidentifiers for destination locations, and possibly variouscharacteristics about destination locations, such as, but not limitedto, whether they are parking spaces, “no parking” locations, and time-and/or date-based restrictions on use.

In an embodiment, when a vehicle appears in the field of view ofdestination camera 125, server system 140 performs two tasks. For thefirst task, server system 140 may be configured to: (1) be aware of anyand all vehicles present in the field of view of destination camera 125;(2) distinguish moving from stationary vehicles; (3) identify a time atwhich a moving vehicle becomes stationary in an area of the field ofview corresponding to a destination location; (4) identify whichdestination location(s) each stationary vehicle is occupying; (5) keeptrack of how long each vehicle has occupied its respective destinationlocation(s); (6) identify a time at which a stationary vehicle vacatesan area of the field of view corresponding to a destination location;(7) determine when use of a destination location conflicts withrestrictions established for the destination location; (8) identifywhich street or block a vehicle is improperly parked; (9) track anamount of time each vehicle has been improperly parked; and (10) track atime at which a vehicle ceases to be illegally parked. Various types ofdestination locations, each having respective restrictions, for wheeledmotor vehicles include, but are not limited to: no parking areas(anytime, fire lane, fire hydrant), no parking during specified timesand/or days (for example, during rush hour or street cleaning), noparking/stopping/standing, limited-time parking (for example, 2 hourtime limit), parking restricted to classes of vehicles (taxi stand, busstop, loading zone/commercial vehicles only), permitted parking(handicapped or resident-only parking), pick-up and drop-off onlylocations, and street intersections (for anti-gridlock “block the box”moving violations in which a vehicle remains within an intersectionduring a red light for the vehicle's direction of travel). Server system140 may be configured to detect other parking violations, such as, butnot limited to, parking within a specified distance of an intersection,parking in a bike lane or crosswalk or other pedestrian zone, not withina space for a single destination location, more than a specifieddistance from a curb, parking in or in front of a driveway, use of anisland or center strip, vehicle parked facing the wrong direction,abandoned vehicle (stationary for more than a specified number of days),oversize vehicles, incorrect class of vehicle (such as a camper,trailer, or boat for more than a specified number of hours in a givennumber of days). A destination location may also be subject to areservation, where an end user reserves exclusive use of a destinationlocation for a specified period of time. Server system 140 may also beconfigured to detect double parked vehicles, while distinguishingvehicles stopping in traffic lanes due to driving conditions fromimproperly double parked vehicles. Server system 140 may also beconfigured to recognize, and in some cases distinguish, smaller wheeledvehicles such as motorcycles, scooters, and “smart cars.”

In the second task the vehicle is “tracked” as it moves across the fieldof view of destination camera 125. There is overlap between an imagecaptured by destination camera 125 and a neighboring identification ordestination camera such that a vehicle is within the field of vision ofboth cameras for a minimum of 2 seconds at a speed of 15 MPH(approximately 44 feet). In general, neighboring camera positions aredetermined to ensure this overlap occurs. As the two cameras haveuniquely different angles of view, obtaining overlapped images of anygiven vehicle from both of these cameras is treated as a positiveidentification of the vehicle (via license plate number obtained fromthe identification image 121) along with the exact parking space thevehicle has occupied (in the event the vehicle parks).

In some configurations, a destination camera may be positioned tomonitor vehicle movement in a particular area, where the area does notinclude any destination locations of interest to server system 140. Suchan area may be located, for example, between two areas each containingdestination locations of interest. In this example, images captured bythe destination camera may overlap images captured by camerascorresponding to these two areas, and can be used for a “handoff”procedure as illustrated in FIG. 4, discussed below.

In an embodiment, destination camera 125 may have pan, tilt, and/or zoomcapability, which may be used for server system 140 to obtain a moredetailed view of a destination location, or may be used by parkingfacility operator personnel via a user interface on parking operationsystem 190 to monitor particular areas of interest viewable bydestination camera 125. In such an embodiment, server system 140 may beconfigured to perform a coordinate transformation in order to monitorusage of destination locations and accurately determine vehiclecharacteristics while varying the field of view of destination camera125.

In an embodiment, preliminary motion detection may be used to reduce thenumber of destination images which are captured and/or analyzed, asthere may be significant periods of time during which no vehicles passthrough the field of view of destination camera 125.

In an embodiment, accurate date and time stamps are included in allimages captured by identification cameras and destination cameras tofacilitate later manual review of stored images, should such a needarise.

In an embodiment, some amount of “grace period” is given to vehiclesafter they are determined to have become stationary within anappropriate destination location (in contrast to an inappropriatedestination location, such as a no parking zone). When a driver leavesthe vehicle to pay for parking, even though it has been determined thatthe vehicle has begun use of a destination location with no paymentmade, some grace period is needed to delay the determination that thevehicle is in violation of a paid use requirement since some amount oftime is required to complete the payment process.

FIG. 4 illustrates how images captured by cameras with overlappingfields of view may be used to identify a vehicle, track the movement ofthe vehicle to a destination location, and identify use of thedestination location by the vehicle. FIG. 4 is not presented to scale;for example, overlapping area 430 a nominally would extend 44 feet ormore in the vertical direction, but it is illustrated as approximatelythe same size as vehicle 410. First, at time t1 an identification camera(not illustrated) captures identification image 420 covering the fieldof view illustrated in FIG. 4. Included in identification image 420 isvehicle 410 at the location designated 410′. Based on image 420 a (andperhaps additional images captured by the identification camera prior totime t1), server system 140 determines a unique identifier for vehicle410. For example, by performing OCR on one or more detected images of alicense plate to obtain a license plate number from which the uniqueidentifier can be obtained. Server system 140 records in database 141,in connection with the unique identifier that vehicle 410 was observedby the identification camera at time t1. In an alternative embodiment,server system 140 instead stores in volatile memory that vehicle 410 wasobserved by the identification camera at time t1, and such informationis recorded in database 141 after a determination that vehicle 410 hasmade use of a destination location prior to vehicle 410 being identifiedby server system 140 via identification images obtained from anidentification camera other than the identification camera whichcaptured identification image 420.

At time t2, the identification camera captures another identificationimage 420 a′ of vehicle 410 at the location designated 410 a. Atapproximately time t2, a first destination camera captures destinationimage 420 b of vehicle 410 within overlap area 430 a (for example, atapproximately the location 410 a). In some embodiments, theidentification camera and the first destination camera are synchronized,such that images 420 a′ and 420 b are both captured at essentially thesame time t2. Overlap area 430 a corresponds to the field of view ofidentification image 420 a′ which overlaps the field of view ofdestination image 420 b. In both images 420 a′ and 420 b, images ofvehicle 410 have been captured while vehicle 410 is within overlap area430 a.

Based on one or more identification images captured by theidentification camera (such as identification image 420 a′), serversystem 140 determines a plurality of characteristics Ca for vehicle 410.As noted above, fixed characteristics for vehicle 410 and recorded indatabase 141 may be included in Ca. Examples of the characteristicsinclude, but are not limited to: a speed of travel for vehicle 410, adirection of travel for vehicle 410, a position of a vehicle in avehicle image (including, for example, a lane of traffic vehicle 410 istraveling in), a color of vehicle 410, and a size or dimension ofvehicle 410. Some characteristics may be considered staticcharacteristics of vehicle 410, and are not expected to change over time(for example, vehicle color or size). Other characteristics may beconsidered dynamic characteristics of vehicle 410, and may changesignificantly over time (for example, vehicle speed or lane of travel).In an embodiment, static characteristics may be recorded in database 141as fixed characteristics of vehicle 410 in connection with the uniqueidentifier determined based on identification image 420 a. These fixedcharacteristics may be determined based on vehicle information providedby an end user (such as make, model, year, and color), or they may bebased on vehicle images captured during a previous encounter betweenvehicle 410 and server system 140. The fixed characteristics may be usedin connection with determining a unique identifier for vehicle 410, byensuring that static characteristics determined from identificationimages correspond to the fixed characteristics. The fixedcharacteristics may also be included in Ca or other pluralities ofcharacteristics where a vehicle identification has been confirmed. Basedon destination image 420 b, and possibly other destination imagescaptured by the first destination camera, server system 140 determines aplurality of characteristics Ca′ for vehicle 410. Due to factors such asthe different angles of view of vehicle 410 for the identificationcamera and the first destination camera, and obstructing vehicles, Caand Ca′ may consist of different, but typically overlapping, sets ofvehicle characteristics. For example, a vehicle width may be determinedfor Ca but not Ca′ due to a partial obstruction of vehicle 410 from thepoint of view of the first destination camera around time t2.Additionally, image distortion, which is typically more significant atthe borders or images, or other factors may result in some vehiclecharacteristics, such as speed or size, only being approximate values.Normalization of characteristics such as size, speed, and direction oftravel may be performed to compensate for camera positioning and zoomlevel.

Server system 140 then determines whether Ca′ corresponds to Ca bycomparing the values of the vehicle characteristics included in both Caand Ca′. As vehicle characteristics determined by server system 140 maybe approximations of the actual characteristics of vehicle 410, anapproximate equivalence between values, where such equivalence isdemonstrated for all of the compared characteristics, is sufficient tosupport a determination that Ca′ corresponds to Ca. If Ca′ is determinedto correspond to Ca, then server system 140 records in database 141, inconnection with the unique identifier, that vehicle 410 was observed bythe first destination camera at time t2. In an alternative embodiment,server system 140 instead stores in volatile memory that vehicle 410 wasobserved by the first destination camera at time t2, and suchinformation is recorded in database 141 after a determination thatvehicle 410 has made use of a destination location prior to vehicle 410being identified by server system 140 via identification images obtainedfrom an identification camera other than the identification camera whichcaptured identification image 420. This avoids writing vehicle locationinformation to database 141 that is not required to demonstrate that ause of a particular destination location by vehicle 410.

At time t3, the first destination camera captures another destinationimage 420 b′ of vehicle 410 at the location designated 410 b. Atapproximately time t3, a second destination camera captures destinationimage 420 c of vehicle 410 within overlap area 430 b (for example, atapproximately the location 410 b). In some embodiments, the firstdestination camera and the second destination camera are synchronized,such that images 420 b′ and 420 c are both captured at essentially thesame time t3. Overlap area 430 b corresponds to the field of view ofdestination image 420 b′ which overlaps the field of view of destinationimage 420 c. In both images 420 b′ and 420 c, images of vehicles 410 and440 have been captured while vehicles 410 and 440 are within overlaparea 430 b.

Based on one or more destination images captured by the firstdestination camera (such as identification image 420 b′), server system140 determines a plurality of characteristics Cb for vehicle 410. Asnoted above, fixed characteristics for vehicle 410 and recorded indatabase 141 may be included in Cb. Based on destination image 420 c,and possibly other destination images captured by the second destinationcamera, server system 140 determines a plurality of characteristics Cb′for vehicle 410. Additionally, due to the inclusion of vehicle 440 indestination image 420 c, a plurality of vehicle characteristics Cx aredetermined for vehicle 440. Differences in dynamic vehiclecharacteristics between Cb′ and Cx would result due to vehicle 440traveling in the opposite direction in a different lane of travel.Server system 140 then determines whether Cx corresponds to Cb, whichshould not occur. Server system 140 then determines whether Cb′corresponds to Cb. If Cb′ is determined to correspond to Cb, then serversystem 140 records in database 141, in connection with the uniqueidentifier, that vehicle 410 was observed by the second destinationcamera at time t3. In an alternative embodiment, server system 140instead stores in volatile memory that vehicle 410 was observed by thesecond destination camera at time t3, and such information is recordedin database 141 after a determination that vehicle 410 has made use of adestination location prior to vehicle 410 being identified by serversystem 140 via identification images obtained from an identificationcamera other than the identification camera which capturedidentification image 420.

At time t4, the second destination camera captures another destinationimage 420 c′ of vehicle 410 at the location designated 410 c. Atapproximately time t4, a third destination camera captures destinationimage 420 d of vehicle 410 within overlap area 430 c (for example, atapproximately the location 410 c). In some embodiments, the seconddestination camera and the third destination camera are synchronized,such that images 420 c′ and 420 d are both captured at essentially thesame time t4. Overlap area 430 c corresponds to the field of view ofdestination image 420 c′ which overlaps the field of view of destinationimage 420 d. In both images 420 c′ and 420 d, images of vehicle 410 havebeen captured while vehicle 410 is within overlap area 430 c.

Based on one or more destination images captured by the seconddestination camera (such as identification image 420 c′), server system140 determines a plurality of characteristics Cc for vehicle 410. Asnoted above, fixed characteristics for vehicle 410 and recorded indatabase 141 may be included in Cc. Based on destination image 420 d,and possibly other destination images captured by the third destinationcamera, server system 140 determines a plurality of characteristics Cc′for vehicle 410. Server system 140 then determines whether Cc′corresponds to Cc. If Cc′ is determined to correspond to Cc, then serversystem 140 records in database 141, in connection with the uniqueidentifier, that vehicle 410 was observed by the third destinationcamera at time t4. In an alternative embodiment, server system 140instead stores in volatile memory that vehicle 410 was observed by thethird destination camera at time t4, and such information is recorded indatabase 141 after a determination that vehicle 410 has made use of adestination location prior to vehicle 410 being identified by serversystem 140 via identification images obtained from an identificationcamera other than the identification camera which capturedidentification image 420. In an embodiment, a second identificationcamera may be responsible for capturing image 420 d, but is unable toobtain an image of a license plate for vehicle 410, in which casecharacteristics of vehicle observed via the second identification cameraare used to confirm that the vehicle observed in overlap area 430 caround area 410 c is vehicle 410. This illustrates that it is notsignificant whether it is a destination camera or an identificationcamera which is used for tracking vehicle 410 as it moves away from theinitial identification camera. When used in connection with suchtracking, these cameras may be broadly referred to as “trackingcameras.”

At time t5, the third destination camera captures another destinationimage 420 d′ of vehicle 410 at the location designated 410 d. Atapproximately time t5, a fourth destination camera captures destinationimage 420 e of vehicle 410 within overlap area 430 d (for example, atapproximately the location 410 d). In some embodiments, the thirddestination camera and the fourth destination camera are synchronized,such that images 420 d′ and 420 e are both captured at essentially thesame time t5. Overlap area 430 d corresponds to the field of view ofdestination image 420 d′ which overlaps the field of view of destinationimage 420 e. In both images 420 d′ and 420 e, images of vehicle 410 havebeen captured while vehicle 410 is within overlap area 430 d.

Based on one or more destination images captured by the thirddestination camera (such as identification image 420 d′), server system140 determines a plurality of characteristics Cd for vehicle 410. Asnoted above, fixed characteristics for vehicle 410 and recorded indatabase 141 may be included in Cd. Based on destination image 420 e,and possibly other destination images captured by the fourth destinationcamera, server system 140 determines a plurality of characteristics Cd′for vehicle 410. Server system 140 then determines whether Cd′corresponds to Cd. If Cd′ is determined to correspond to Cd, then serversystem 140 records in database 141, in connection with the uniqueidentifier, that vehicle 410 was observed by the fourth destinationcamera at time t5. In an alternative embodiment, server system 140instead stores in volatile memory that vehicle 410 was observed by thefourth destination camera at time t5, and such information is recordedin database 141 after a determination that vehicle 410 has made use of adestination location prior to vehicle 410 being identified by serversystem 140 via identification images obtained from an identificationcamera other than the identification camera which capturedidentification image 420.

In association with the field of view for image 420 e illustrated inFIG. 4, four appropriate destination locations 450 a to 450 d (such asparking spots with an hourly rate for use) have been specified to serversystem 140. At time t6, destination image 420 e′ is captured by thefourth destination camera, at a time when vehicle 410 is at location 410e. Based on destination image 420 e′, server system 140 determines thatvehicle 410 is within the rectangular region specified for destinationlocation 450 b. At time t7, destination image 420 e″ is captured by thefourth destination camera, with vehicle 410 remaining at location 410 e.Based on destination image 420 e″, server system 140 again determinesthat vehicle 410 is within the rectangular region specified fordestination location 450 b, accordingly determines that vehicle 410began use of destination location 450 b beginning at time t6, andrecords this information in database 141. At time t8, destination image420 e′″ is captured by the fourth destination camera, at a time whenvehicle 410 has exited destination location 450 b. Based on destinationimage 420 e′″, server system 140 determines that vehicle 410 is nolonger within the rectangular region specified for destination location450 b, accordingly determines that vehicle 410 completed its use ofdestination location 450 b at time t8. Based on information as torestrictions applicable to use of destination location 450 b from thetime period t6 to t8, such as, for example, an hourly rate for use ofthe destination location 450 b, server system 140 can initiate actionssuch as billing an end user account for this use of destination location450 b, or such as levying a fine for improper use of destinationlocation 450 b.

Server system 140 is also configured to correctly identify a vehiclewhich has made use of a destination location where server system 140first determines that a vehicle has made use of a destination location,and then later is able to capture a vehicle identification via anidentification camera. This situation may occur where, for example, aview of a vehicle is obstructed as it passes by an identificationcamera, or a restart of server system 140 results in identification ofthe vehicle not being made previously. As an example, if at time t0,before time t1, the fourth destination camera was to first determinethat vehicle 440 was making use of destination location 450 c, but nothave an identification for vehicle 440, a temporary unique identifierwould be assigned to vehicle 440, and entries recorded in database 141in connection with the temporary identifier. Sets of vehiclecharacteristics would be determined for vehicle 440 while it made use ofdestination location 450 c, as well as when it passed through overlapareas 430 d, 430 c, 430 b, and 430 a, and the correspondence of thesesets of characteristics would be performed much in the same mannerdescribed above, to determine and record that the same vehicle 440associated with the temporary identifier, was still being observed. Onceobserved by the first identification camera while located in overlaparea 430 a, server system 140 would be able to determine a correctunique identifier for vehicle 440. At that point, records previouslyrecorded in database 141 under the temporary identifier can be modifiedor stored again under the correct unique identifier for vehicle 440,which the use of destination location 450 c properly associated withthat identifier. An alternative mechanism for obtaining anidentification of vehicle 440 after time t0 is to dispatch mobile device150 to capture one or more identification images of vehicle 440 while itis making use of destination location 450 c. Dispatch of mobile device150 may be designated a higher priority where the use of destinationlocation 450 c by vehicle 440 is determined to conflict withrestrictions associated with use of destination location 450 c.

Server system 140 receives images from the identification anddestination cameras and interpolates the identities of parked vehicles.Server system 140 may then retrieve parking payment information (suchas, but not limited to, duration of paid parking and time of expiration)from a third-party parking payment system via an API (applicationprogramming interface). If a vehicle parking status is determined to beeither unpaid or expired, the server system 140 transmits thisinformation to the third-party parking payment system via an API. Theparking facility operator which runs the third-party payment system canthen follows its Standard Operating Procedures (SOP) in dealing withparking violations (such as, but not limited to, mailing a parkingviolation ticket, installing a wheel clamp on the vehicle, or towing thevehicle upon repeated or severe parking violations).

Vehicle detection by server system 140 may be configured to performseveral functions: (1) detecting the presence and location of vehiclesin video frames; (2) distinguishing moving vehicles from stationaryvehicles; (3) identifying whether particular detected vehicles areoccupying destination locations, such as parking spaces or no-parkingareas; (4) determine a time at which a vehicle begins to occupy orvacates a destination location; (5) track vehicles as they travel fromthe field of view of one tracking camera to another, and raise anexception if an error in this tracking is determined to have occurred.In an embodiment, server system 140 may also be configured to identifywhether particular vehicles are displaying an appropriate indicator foraccess or use of a destination location. Examples of such indicatorsinclude, but are not limited to, a handicapped parking hanger or decal,or a parking hanger or decal indicating authorized use of a “residentonly” parking location. A camera may be zoomed in to more clearlycapture images of such indicators. In an embodiment, a vehicleidentifier, such as a license plate, may be used to determineappropriate access or use of a destination location, and obviate theneed for a separate indicator.

In an example implementation, server system 140 may be programmed to usethreads to carry out its tasks for identification and destinationcameras. For example, a “video_receiver” thread may handle the receptionof a video and/or image stream from an identifier or destination cameraand store raw video and/or image information. This may be with onethread per video and/or image stream or multiple video and/or imagestreams. Configurable options for this thread may include (1) store tofile, (2) store to memory, (3) store to memory and file, (4) a retentiontime for video data (for example, 4 hours)). Another example is a threadto detect vehicle color, shape, location, lane position, speed/directionof travel, whether a vehicle location is in a handoff zone (and whichone), camera identification, and storing such information in database141. This thread would examine each multiple video frames to determineupdated locations and speed/direction of travel of vehicles and storingsuch information in database 141. There may be one thread per videoand/or image stream or multiple video and/or image streams. Anotherexample is a thread which confirms a particular vehicle detected asexiting via a handoff zone is detected by a nearby camera. This may bedone, for example, by reviewing the database 141 for a particular camerabased on in which handoff zone the vehicle was detected and/or adetected direction of travel for the vehicle. There may be one threadper video and/or image stream or multiple video and/or image streams.Another example is a thread to determine whether a vehicle is occupyinga destination location by remaining stationary at that destinationlocation, along with a time when vehicle begins to occupy or vacates thedestination location. There may be one thread per video and/or imagestream or multiple video and/or image streams.

Many machine vision techniques are known in the art which may be usedfor detecting the presence of vehicles in video frames. For example,there are various known edge detection algorithms which may be used fordetecting the presence of vehicles. As an example, an algorithm maydetect the presence of a vehicle based on differences in color betweenthe vehicle and the background, and whether a size of a total shape islarger than a minimum vehicle size expected in consideration of zoomlevel and/or location in a field of vision. For example, the algorithmmay compare colors of pixels in a video stream snapshot against eachother. Since an image likely includes various non-vehicle objects suchas a fire hydrant, pedestrian, grass, road markings, and paint, thealgorithm may detect groups of pixels with similar colors (with aconfigurable range of variation in color considered to be a match). Thenumber and location of pixels in the total shape are used to determinewhether the shape is larger than the minimum vehicle size. The algorithmmay further accommodate “holes” or imperfections in the shape, which mayoccur due to specular reflections or highlights, for example.Additionally, the algorithm may determine whether a number of shapes ofrespective minimum sizes are within certain distances of each other.These shapes may correspond to, for example, hood, roof, and trunkportions of a vehicle, which are typically of the same color.

A boundary condition may occur where there is insufficient contrast ordifference in color between a vehicle and the background, in whichdetection may fail. For example, a black vehicle against black asphalt.Further refinements, to camera hardware and/or the algorithm, may beintroduced to address this boundary condition. For example, a camerathat performs both visible light and thermal imaging may be used, whichobtains a heat signature of a vehicle that has begun occupying adestination location. In this example, the engine compartment would havea different thermal profile than its surrounding even though both thevehicle and the backdrop are black in color. Further cost-benefitanalysis would determine whether the benefits gained from such anapproach justifies the incremental cost, as well as taking in accountany boundary conditions caused by the thermal imaging process can besufficiently addressed (such as by the visible light captured by thecamera).

Although the presence and many characteristics of a vehicle can berecognized from a single image, multiple images are needed and used todetermine, for example, if the vehicle is stationary or moving. In anexample, server system 140 may be configured to: (1) based ondeterminations of color and size of vehicles in the current image, findcorresponding matches in previous images (for example, by reviewing datastored in database 141 or memory); and (2) once such matches are found,compare the locations of pixels between the current image and previousimages, taking into account any change in zoom level. Based on suchprocessing, one or more of the following scenarios may be detected. (A)if locations of pixels are unchanged or have unchanged below apredetermined threshold, all vehicles are deemed to have remainedstationary. In this scenario, a simple update of the date and time isperformed for existing entries in database 141 or memory for thevehicles. (B) if pixels associated with one or more vehicles are at newlocations, then the one or more vehicles are determined to have moved.Server system 140 may calculate speed and direction of travel for thesevehicles and update database 141 or memory with this information, alongwith updated locations. (C) if one or more vehicles have disappearedfrom the current image, server system 140 may be configured to removedata recorded in database 141 or memory for such vehicles. (D) if one ormore vehicles have newly appeared in the current image, data associatedwith the new vehicles may be recorded in database 141 or memory. In anembodiment, server system 140 may determine that one of the new vehiclescorresponds to a vehicle recently determined to have disappeared, andmay treat the view of the vehicle as having been temporarily obstructed.(E) if all vehicles have disappeared from the current image, data fromdatabase 141 or memory from the preceding interval may be removed inconnection with the vehicles therein. However, some data may be retainedin order to perform vehicle tracking to or from an identificationcamera, as discussed previously.

In an example, an algorithm for identifying which vehicles are occupyingdestination areas is similar to the basic vehicle detection algorithm.Once vehicle detection is completed, server system 140 evaluates whetherpixel locations corresponding to a vehicle occupy any of the pre-mappedvisual regions. A degree of overlap may be configurable. In anembodiment, a vertical offset may be applied to compensate for vehicleheight and compensate for a visual obstruction, such as a parkedvehicle, between a destination camera and the vehicle. Such obstructionsmay frequently occur in dense parking lots, for example. Also, serversystem 140 determines whether the vehicle has remained stationary for atleast a certain amount of time. Once it is determined that a vehicle hasdwelled in a given destination location for at least the certain amountof time, identification information for the vehicle, the destinationlocation, and a time and date at which the vehicle began to make use ofthe destination are recorded in database 141, preferably in non-volatilestorage. In an embodiment, server system 140 may be configured to detectwhen a vehicle takes up multiple destination locations, such as byoccupying two parking spaces. This may result in, for example, issuanceof one or more parking citations or increased usage fees. Server system140 may also determine when the vehicle vacates the destinationlocation, and record corresponding information in database 141. Also,server system 140 may be configured to determine when user of thedestination location by the vehicle will soon exceed, or has exceeded, aperiod of time for use of the destination location, such as may occur inconnection with a parking spot with a two-hour maximum parking duration,a destination location not available for vehicle use after a specifiedtime for street sweeping, or the exhaustion of funds provided for thevehicle.

In an embodiment, server system 140 may be configured to recognize oneor more multi-vehicle destination locations, which may be used by morethan one vehicle at a time. For example, server system 140 may beconfigured to support tracking for “long block” parking of vehicles, inwhich a parking area controlled by, for example, a single Pay-by-Space,Pay-and-Display, or Pay-by-Plate unit is provided along the length of acurb for a city block, but the parking area is not divided into markedparking spaces; instead, vehicles can make use of any available spacealong the curb (excluding certain “no parking” regions, such as, but notlimited to, loading zones and driveways). In another example, serversystem 140 may be configured to track use of a parking lot, but not aplurality of individually marked spaces therein, controlled by, forexample, a single Pay-and-Display unit. Although server system 140 maynot be configured to track use of a multi-vehicle destination locationon a space by space basis, server system 140 may determine and recordthe position of a vehicle within the multi-vehicle destination locationfor tracking the vehicle's use over time and for parking enforcementactivities, such as attaching a citation to a vehicle which has exceededits permitted use of the multi-vehicle destination location. In anembodiment, server system 140 may be configured to determine a length ofa vehicle making use of a multi-vehicle destination location, as agreater parking fee and/or fine may imposed for the use of space thatotherwise could be occupied by more than one vehicle. One benefit ofusing the server system 140 described in this application is thatwhereas conventional controlled parking lots might rely on a singlepoint of ingress and egress for controlling use of the lot, tracking byuse of cameras allows for a parking lot with multiple entrances andexits and a more “open” design, as vehicles do not need to be confinedby wall of fencing.

In an embodiment, multiple noncontiguous destination locations ormulti-vehicle destination locations may be aggregated into a singlemulti-vehicle destination location. This may include where twoconstituent destination locations are covered by different destinationcameras. This may be useful where, for example, a multi-vehicledestination location is broken up by various “no parking” destinationlocations along the length of a city block.

In an embodiment, there may be a restricted destination location (suchas a “no parking” overlaying a multi-vehicle destination location, so asto “carve out” restricted areas of the multi-vehicle location. Forexample, in the “long block” scenario discussed above, a singlemulti-vehicle destination location may be defined along the entirelength of a city block, with “no parking” destination locationsoverlaying various portions along the city block, such as, but nolimited to, loading zones and driveways along the length of the block. Avehicle detected by server system 140 as making use of one of the “noparking” destination locations will be determined by server system 140as having committed a parking violation, despite the vehicle also havebeen within the larger multi-vehicle destination location.

In an embodiment, a computer-based configuration utility may beprovided. For example, mobile device 150 or manual review system 180 maybe configured with a configuration utility, allowing for on-site orremote configuration. For example, mobile device 150 may be used by aperson performing camera installation, in order to provide feedback asto the effectiveness of a camera configuration for vehicleidentification, tracking, and/or detection. Alternatively or inaddition, certain system or camera configuration changes may be made viamanual review system 180 as a result of configuration shortcomingidentified during use of manual review system 180. In an embodiment, auser interface may be made available via a web browser. Theconfiguration utility may provide one or more features such as, but notlimited to:

-   -   Providing a graphical user interface (GUI) to perform        calibration of vehicle detection for a given camera.    -   Providing a GUI for ensuring contiguous camera alignment.    -   Providing a GUI for specifying minimal vehicle dimensions. This        may be in terms of pixels and adjusted for zoom level.    -   Providing a GUI for identifying a portion of a camera's field of        view which overlaps with the field of view of another camera.    -   Providing a GUI for aligning a camera and/or specifying camera        orientation and/or alignment.    -   Providing a GUI for pre-mapping destination locations.    -   Defining circumstances which are considered vehicle parking        violations. Such circumstances may be particular to a given        region or location.    -   Configure outcomes for various parking violations. For example,        information about a violation may be simply displayed on a        screen, registered owner information may be obtained for issuing        a citation or bill, or a warning may be printed and placed on a        vehicle.    -   Configuring an expiration period after which a new request for        vehicle owner registration information must be made, such to a        state-run motor vehicle agency, before issuing a parking        citation by way of mail or otherwise.    -   Provide a GUI for other system configuration parameters to be        reviewed and modified.

To perform calibration of vehicle detection for a given camera, a GUIpresents two image feeds and allows an administrator to superimposerulers on each image feed. A zoom level, if any, may be displayed aspart of the GUI, as may be a control for modifying the zoom level. Thetwo image feeds originate from the given camera and another camera withan overlapping field of vision (such cameras are often adjacent to eachother). Preferably, for calibration the capture of images for the twocameras is synchronized such that images are captured at the same time,allowing for review of a vehicle's position in each video feed, such aswhen it passes through the overlapping area of the two cameras' fieldsof vision. The configuration utility may also provide, by way of theGUI, temporal image control functions such as pause, play, rewind, fastforward, freeze frame advance, freeze frame reverse, and goto framefunctions.

For calibration, the configuration utility may be used to find one ormore vehicles of known make and model in one or both image feeds (thismight include, for example, a vehicle used by a technician operating theconfiguration utility), and position one or more rulers in the GUI suchthat the rulers indicate the correct length, width, and/or height of thevehicles by reference to known values for these dimensions for thevehicles being measured. In an embodiment, the ruler function providedby the GUI is configured to compensate for nonlinearities due to factorssuch as visual distortion introduced by the camera (which may have knowncharacteristics based on a camera model, for example) or the perspectiveof the captured image. For example, ruler markings indicating units ofdistance may be spaced closer on a right-hand side of an image wherevehicles captured in that area are further away from the camera thanthose captured in the left-hand side of the image. The configurationutility may be configured to allow for on-site calibration of suchnonlinearities for a given camera by identifying markers in an image,and specifying positions and/or distances (relative to other markersand/or the camera). Such markers may be temporarily, or in some casespermanently for the purpose of later recalibration, placed by atechnician.

Also, the configuration utility may be used to specify relationshipsbetween directions of travel in the two image feeds, as well as identifypixels or image locations corresponding to common points in the twoimage feeds. In an embodiment, locations and directions of travel may bespecified for lanes/paths of traffic in each of the image feeds, andcorrelated between the two images feeds (allowing, for example, laterdetermination of when a vehicle passes from the field of one camera toanother within the same lane/path of traffic). Specification of thesefeatures may be done via the GUI, for example by allowing for theoverlay of lines or other indicators on the image feeds indicatinglane/path boundaries. FIGS. 6A and 6B illustrate aspects of an exampleGUI. FIG. 6A illustrates a portion of the GUI in which an image 600 afrom a first camera is displayed. FIG. 6B illustrates a portion of theGUI in which an image 600 b from a second camera is displayed. The fieldof views of the images 600 a and 600 b overlap, with the area of overlapincluding areas 640 a and 640 b. Also captured in images 600 a and 600 bare two lanes/paths of traffic, labeled as lanes/paths 611 a and 612 ain FIG. 6A, and lanes/paths 611 b and 612 b in FIG. 6B. In thisparticular example illustrated in FIGS. 6A and 6B, traffic proceeds inthe same direction (generally corresponding to angle θ) in lanes/paths611 a and 611 b, although in a similar example traffic might proceed inopposing directions in lanes/paths 611 a and 611 b. Also captured inimages 600 a and 600 b are images of three vehicles, respectivelylabeled A, B, and C. In FIG. 6A, lane/path boundaries have beenspecified with the dashed lane/path boundary indicators 621 a, 622 a,and 623 a, which may be added, placed, moved, labeled, and otherwiseidentified by use of the GUI. In FIG. 6A, two traffic lanes/paths havebeen specified using the GUI: the lane/path 611 a between lane/pathboundary indicators 621 a and 623 a, and the lane/path 612 a betweenlane/path boundary indicators 622 a and 623 a. In the particular exampleillustrating in FIG. 6A, traffic proceeds in lanes/paths 611 a and 612 ain the same direction 8, and lane/path boundary indicators 621 a and 622a are of a first type (illustrated by the similar dashed lines), andlane/path boundary indicator 623 a is of a second type indicating thedivision of the area of unidirectional traffic flow between lane/pathboundary indicators 621 a and 622 a into the two lanes/paths 611 a and612 a. The configuration utility may be configured to automaticallyprovide or suggest identifiers for the specified lanes for crossreferencing with, for example, lanes identified in connection with othercameras, such as lanes/paths 611 b and 612 b illustrated in FIG. 6B. InFIG. 6B, the configuration utility provides similar facilities forlane/path identification, with lanes/paths 611 b and 612 b, andlane/path boundary indicator lines 621 b, 622 b, and 623 b correspondingto their counterparts in FIG. 6A described above.

In an embodiment, each lane/path of traffic may be divided into a numberof smaller segments. These segments may be specified via the GUI, forexample by allowing for the overlay of lines or other indicators on theimage feeds indicating the segments. In an embodiment, the overlaidlines may be used to aid in automatically characterizing nonlinearitiesin image feeds. FIG. 6A illustrates an example, with lane/path segmentindicator 631 a and other similar unlabeled lane/path segmentindicators. The configuration utility may be configured to automaticallygenerate suggested lane/path segment indicators based on previouslyspecified lane/path boundary indicators, and provide via the GUI toolsfor adjusting the lane/path boundary indicators as desired. Also, theinclusion of vehicle images, such as vehicles A, B, and C shown in FIG.6A, may aid a technician in accurately placing or adjusting lane/pathsegment indicators. FIG. 6B illustrates lane/path segment indicator 631b and other similar unlabeled lane/path segment indicators correspondingto their counterpart in FIG. 6A described above.

The configuration utility may also provide similar features via the GUIfor specifying an area of an image stream with an overlapping field ofview with a respective area of a second image stream. As these areas areused for “handing off” the detection of a vehicle from one trackingcamera to another, these areas may be referred to as “handoff zones.”For example, FIGS. 6A and 6B illustrate images captured by cameras withoverlapping fields of view, in which the field of view for handoff zone640 a overlaps the field of view for handoff zone 640 b. The GUI may beconfigured to allow for specification of handoff border indicators 651 aand 652 a in FIG. 6A, and respective handoff border indicators 651 b and652 b in FIG. 6B. Also, a specified handoff zone may be divided into anumber of smaller segments, using handoff segment indicators 611 a and611 b (and other unlabeled handoff segment indicators) illustrated inFIGS. 6A and 6B. The configuration utility may be configured toautomatically generate suggested handoff segment indicators based onpreviously specified handoff boundary indicators, and provide via theGUI tools for adjusting the handoff boundary indicators as desired.Also, the inclusion of vehicle images, such as vehicles A, B, and Cshown in FIG. 6A, may aid a technician in accurately placing oradjusting handoff segment indicators. In an embodiment, theconfiguration utility may be configured to determine a distance betweenhandoff border indicators 651 a and 652 a, an alert a user of the GUI ifthis distance is below a minimum desired distance.

Although the GUI illustrated in FIGS. 6A and 6B shows and describesindicators provided by the GUI as simple lines, the configurationutility and the GUI may also provide for specifying features with morecomplex shapes, such as, but not limited to, curves and freehand lines.Also, where an indicator specified in connection with a first camera,such as lane/path boundary indicator 623 a, has a counterpart indicatorin connection with another camera, such as lane/path boundary indicator623 b, these counterpart indicators may be linked using the GUI. In anembodiment, such links may be automatically determined or suggestedbased on information such as previously specified indicators, a knownposition of a camera, a known orientation of the camera, and knownaspects of traffic flow proximate to a camera, such as numbers oflanes/paths and directions of traffic flow within those lanes/paths.

In an embodiment, where a camera provides for zoom and/or pan/tiltcapabilities, the configuration utility may be configured to allow forthe specification of various indicator features in images captured withthe camera at various zoom levels or orientations. This additionalinformation may be used to more fully specify the overall field of viewof the camera. Once a technician is satisfied with how indicators and/orrulers have been used to specify the field of view of a camera, thisinformation and/or other information derived therefrom may be stored indatabase 141 for use in, for example, vehicle tracking by server system140.

The configuration utility may be configured to provide a GUI to performcontiguous video camera alignment. This functionality may be providedvia additional user interface elements included in the above-describedGUI for specifying lane/path boundaries. Contiguous video cameraalignment refers to the fact that during an initial camera hardwareinstallation process, or if existing camera hardware is later updated,if the camera is equipped with pan, tilt, and/or zoom (PTZ) function,the pan, tilt, and/or zoom settings of the camera need to be adjustedand/or calibrated to ensure there is sufficient overlap in the field ofvision between cameras, such as the overlap illustrated in FIGS. 4, 6A,and 6B. A technician can review the GUI to determine whether a handoffzone of sufficient size and/or duration is provided by a pair ofcameras. If not, pan, tilt, and/or zoom settings may require changes, orthe physical installation location of a camera may need to be changedaltogether.

The configuration utility may be configured to provide a GUI to performpre-mapping of destination locations, such as parking spaces and “noparking” zones. The GUI presents an image feed and allows a technicianto superimpose grayed out polygons, for example, on an image feed. Inanother nonlimiting example, a parking spot #123 might be mapped to acollection of 20 pixels as captured by a particular destination cameraat a given zoom level and/or PTZ orientation. The GUI may be configuredto provide operations for including, but not limited to, moving,resizing, and rotating such polygons to indicate the position and extentof destination locations within a field of view of a camera. Where acamera has PTZ capabilities, PTZ controls may be provided. Thisfunctionality may be provided via additional user interface elementsincluded in the above-described GUI for specifying lane/path boundaries.The GUI may also provide an interface of assigning an identifier to adestination location which may be used, for example, to provide a commonidentifier for the destination location for use by various aspects ofserver system 140. The GUI may also provide interface elements allowingfor various characteristics of a destination location to be specified,although it may be preferable to employ other aspects of theconfiguration utility or server system 140 to manage, assign, and/ormodify such characteristics with minimal direct human operation.

In an embodiment, the configuration utility may be configured to providea GUI to perform pre-mapping of multi-vehicle destination locations,discussed above, in much the same manner as the above description forpre-mapping single-vehicle destination locations. The configurationutility may be configured to allow multiple destination locations ormulti-vehicle destination locations to be aggregated into a singledestination location. For example, each of the constituent destinationlocations might be assigned a common identifier to indicate they form amulti-vehicle destination location. The configuration utility may beconfigured to allow restricted destination locations to be overlaid amulti-vehicle destination location, to “carve” out portions of a broadarea subject to parking restrictions, such as, but not limited to,loading zones and driveways.

The configuration utility may be configured to provide a GUI forspecifying what constitutes a parking violation. In a first example, agrace period duration may be specified with the GUI, which defines anamount of time a vehicle may occupy a destination location before thevehicle may be considered to be in violation of restrictions on use ofthe destination location. For example, when a vehicle stops at a parkingmeter, it is reasonable to provide a grace period sufficient for thedriver of the vehicle to leave the vehicle and pay for parking.Differing periods of time may be specified for different types or groupsof destination locations (for example, a “no parking” zone may have asmall or zero grace period. In a second example, an amount of vehicleoverlap may be specified with the GUI, as to how much in terms of squarefootage or percent area coverage a vehicle overlaps a destinationlocation before the vehicle may be considered to be in violation. In athird example, permit parking rules may be specified with the GUI,including rules in connection with, for example, the use of handicapparking spaces, resident-only parking, or other parking requiring thedisplay of a permit or decal. The GUI may also provide an interface foridentifying characteristics and features of a required permit or decal,such as, but not limited to, color, an image featured on the permit ordecal, and a location of an identifier (such as a serial number) for thepermit or decal. In a fourth example, a GUI may be configured to definerules for determining what constitutes an expired license plate tag,such as whether a tag indicating the current month is considered to haveexpired.

The configuration utility may be configured to provide a GUI forspecifying consequences for vehicle misuse of a destination location.Examples include, but are not limited to, on-screen display of licenseplate information, license plate lookup to obtain the name and addressof a registered owner, dispatching an enforcement officer, provide alaser-based illumination of a vehicle misusing a destination location,applying a parking fine to a credit card or account on file, anddeducting a parking fine from a checking or other monetary account.

The configuration utility may be configured to receive and recordparameters specifying certain vehicles for which outcomes of detectedparking violations would be modified (for example, a police vehicle maybe identified as exempt from specified violations). For example, suchvehicles may be identified by license plate number, or bear a specifiedidentifier such as a decal.

In an embodiment, server system 140 may be configured to obtainpreexisting street and/or parking information, which may be obtainedthrough a commercial or municipal database and/or provided by a parkingoperator, and automatically determining destination and trafficlane/path information In an embodiment, based on information indicatingthe location and orientation of the various identification anddestination cameras, server system 140 may be configured toautomatically pre-map destination locations and/or traffic lanes/paths.In an embodiment, server system 140 may be configured to record and/orupdate characteristics for destination locations and or trafficlanes/paths recorded in database 141, such as, but not limited to,identifiers, locations (for example, latitude/longitude or streetaddress), rates for use of destination locations, and time-based,date-based, and other restrictions for destination locations describedin this application. In an embodiment, server system 140 may provide anAPI to more easily allow a parking provider to upload destinationlocation information and/or updates in bulk.

In an alternative embodiment, server system 140 and other elementsillustrated in FIG. 1 are in the form of a turnkey platform where nointegration with third-party parking payment systems via an API isrequired. In such an embodiment, part of the turnkey platform consistsof payment stations/equipment, and equipment that generate parkingviolation tickets for mailing.

In an embodiment, onsite payment system 160 may include a turnkeyplatform Pay-by-Phone feature. The end result is that end users can payfor parking by calling a particular IVR (interactive voice response)system-driven phone line and input a destination location identifier orvehicle license plate number and a duration of parking, regardless ofwhether a destination location is associated with a curbside parkingmeter, a Pay-by-Space parking space, or a parking lot.

The turnkey Pay-by-Phone module enables end users to pay for parking byuse of a telephone—typically cellular phone. The turnkey Pay-by-Phonemachine enables database-level integration with server system 140.Database-level integration is superior to API-level integration sincedatabase-level integration offers more flexibility and speed.

The turnkey Pay-by-Phone module receives data from one or more DTMFtransceivers where users enter information over the telephone. There isno requirement for the turnkey Pay-by-Phone module to receive data fromserver system 140 in order for the turnkey Pay-by-Phone module tooperate correctly. The turnkey Pay-by-Phone module may be configured tooutput data to mobile device 150 for manual paid parking validation, aswell as to server system 140 for self-enforcement. All self-enforcementdata processing takes place on server system 140.

In an embodiment, onsite payment system 160 may include Pay-Online on aturnkey platform. The end result is that end users can pay for parkingor view prior history by visiting a website and enter a destinationlocation identifier or vehicle license plate number and the duration ofparking regardless of whether a destination location is associated witha curbside parking meter, a Pay-by-Space parking space, or a parkinglot. This extends to other online conduits such as smart phones and, inthe future, Internet-enabled media centers in cars.

The turnkey Pay-Online module receives data online where end users enterinformation over the Internet. There is no requirement for the turnkeyPay-Online module to receive data from server system 140 in order forthe turnkey Pay-Online module to operate correctly. The turnkeyPay-Online module outputs data to the mobile device 150 for manual paidparking validation, as well as to the server system 140 forself-enforcement. All self-enforcement data processing takes place onserver system 140.

In an embodiment, onsite payment system 160 may include aPay-and-Display machine with QR codes. QR codes have become popular, andaccordingly have significant smartphone support. Further, QR codes caneasily hold all of the relevant information typically contained on aPay-and-Display ticket (such as, but not limited to, parking duration,parking expiration time, amount paid, date, and time). The turnkeyPay-and-Display machine allows end users to pay for parking and leave adisplay ticket with a QR code on the dash of the vehicle. The turnkeyPay-and-Display machine enables database-level integration with serversystem 140. Database-level integration is superior to API-levelintegration since database-level integration offers more flexibility andspeed.

The turnkey Pay-and-Display machine receives payment from end users viaa built-in keypad, coin acceptor, credit card reader, and printer. Thereis also a text/graphics LCD display. There is no requirement for theturnkey Pay-and-Display machine to receive data from server system 140in order for the turnkey Pay-and-Display machine to operate correctly.The turnkey Pay-and-Display machine outputs data to mobile device 150for manual paid parking validation, as well as to server system 140 forself-enforcement. All self-enforcement data processing takes place onserver system 140.

In an embodiment, onsite payment system 160 may include a Pay-by-Spaceturnkey machine. Pay-by-Space is increasingly becoming popular comparedto Pay-and-Display which has a longer history of operation. Parkingoperations using Pay-by-Space are generally more efficient since parkingenforcement personnel no longer have to walk to each vehicle and leaninto the windshield in order to manually check the printed expirationtime. Instead, they can drive in a patrol vehicle and perform visualchecks from a distance to see whether an occupied parking space has beenpaid for. In addition, Pay-by-Space offers enhanced user experiencesince drivers no longer have to walk back to their vehicles to displaythe ticket after paying at the machine—instead, they can continue on totheir destination.

The turnkey Pay-by-Space machine receives payment from end users via abuilt-in keypad, coin acceptor, credit card reader, and printer. Thereis also a text/graphics LCD display. There is no requirement for theturnkey Pay-by-Space machine to receive data from server system 140 inorder for the turnkey Pay-by-Space machine to operate correctly. Theturnkey Pay-by-Space machine may output data to the mobile device 150for manual paid parking validation, as well as to server system 140 forself-enforcement. All self-enforcement data processing takes place onserver system 140.

In an embodiment, onsite payment system 160 may include a Pay-by-Plateturnkey machine. The turnkey Pay-by-Plate machine receives payment fromend users via a built-in keypad, coin acceptor, credit card reader, andprinter. There is also a text/graphics LCD display. There is norequirement for the turnkey Pay-by-Plate machine to receive data fromserver system 140 in order for the turnkey Pay-by-Plate machine tooperate correctly. The turnkey Pay-by-Plate machine may output data tothe mobile device 150 for manual paid parking validation, as well as toserver system 140 for self-enforcement. All self-enforcement dataprocessing takes place on server system 140.

Integration with third-party parking systems is an option rather than arequirement for server system 140. In a turnkey configuration where theentire parking equipment platform is provided, for example, through asingle supplier, no integration with third party parking systems isrequired. However, there are many advantages to the integration approachusing API. An API provides a standard software programming interfacedesigned to allow an external program to gain access to information in ahost system without exposing any trade secrets of the host system. Thisis achieved by creating clearly defined functions, objects, andvariables at the boundary of the two systems, such that the externalsystem is allowed to obtain pre-defined set of information without anyknowledge of how the information is gathered, stored, or calculatedwithin the host system itself. As such, companies generally have noreservations with publishing API for other systems to interface to itsproducts or servers. Data may be exchanged via an API via network-basedmechanisms, often protected by encryption mechanisms such as SSL,relying on XML, JSON, HTML, MIME, or other encodings or languagessupported by the API.

There are two ways for server system 140 to integrate with an API.First, by using an API published by the third-party parking system,server system 140 can gain access to vehicle payment information (suchas, but not limited to, duration paid, and space number). Combined withthe vehicle detection and vehicle identity information server system 140already possesses, server system 140 has or can readily obtain thecomplete information necessary to produce a list of vehicle licenseplates with detailed parking violation information to a parking facilityoperator. The parking facility operator can then follow its SOP, andmail parking violation tickets, install wheel clamps, or tow repeat orsevere offenders.

The second integration method is for server system 140 to publish itsown API so third-party parking systems can obtain information on vehicledetection and vehicle identity. Combined with the vehicle paymentinformation already possessed by the third-party parking system, thethird-party parking system can then provide a seamless presentation tothe parking operator including a list of vehicle license plates withdetailed parking violation information. The third-party parking systemprovider may prefer this approach in order to retain control over howthe parking violation information flows back to the parking operator.

By use of APIs, server system 140 can integrate with the following typesof parking payment systems: Pay-by-Phone, Pay-by-Space, Pay-and-Display,Curbside Parking Meters (single or double space), and Live-OperatorOption.

The basic premise of Pay-by-Phone is an end user (such as a driver of avehicle) calls a phone number displayed in the vicinity of a destinationlocation. Typically the phone number offers IVR (interactive voiceresponse) which guides callers through an automated menu. The end useris prompted to input a unique destination location number, followed byparking duration desired, vehicle license plate number, and credit cardinformation.

Pay-by-Phone is starting to gain traction as a popular “overlay” paymentapproach, where parking facility operators add Pay-by-Phone capabilityto curbside parking meters, Pay-by-Space, and Pay-and-Display machines.The typical motivation is to provide end users an alternate way ofpaying for parking in an effort to increase parking revenue. Forexample, Pay-by-Phone may be added to traditional curbside parkingmeters (single or double space) in order to allow end users to pay withcredit cards over the phone.

In an embodiment, server system 140 may include a Pay-by-Phoneintegration module, which may be a software component that extractsparking payment information from a third party parking payment systemwhich server system 140 uses for parking self-enforcement. Theintegration module for pay-by-phone performs steps including: (1)establishing a connection with a third-party Pay-by-Phone API; (2)issuing a request for data; (3) receiving data; (4) processing andstoring data to database 141; (5) determining parking violation status;(6) marking use of a destination location as a parking violation indatabase 141 for further processing, for example, by the below-describedparking violation pre-processing logic; (7) periodic confirmation thatthe third-party Pay-by-Phone API is active and valid, such as, but notlimited to, by use of “keep-alive”/“ping” messages or commands; and (8)repeating steps 2-7 or closing the connection. In connection with step(1), an API typically specifies the mechanism through which a connectionto it can be established. The Pay-by-Phone integration module isconfigured to observe the protocol required by the third-partyPay-by-Phone API and establishes a connection.

In connection with step (2), when server system 140 determines that aparticular vehicle has vacated a destination location, server system 140signals this event to the Pay-by-Phone integration module, such as via adatabase or Inter-Process Communication (IPC) call, in response to whichthe Pay-by-Phone integration module issues a request to the third-partyPay-by-Phone API for parking payment information for the particularvehicle. Data sent to the third-party Pay-by-Phone API may include, forexample, vehicle license plate information (including plate number andstate/province information), an identifier for the destination locationrecognized by the Pay-by-Phone API (which may require translation froman identifier used internally by server system 140), and a date/time theparticular vehicle vacated the destination location.

In connection with step (3), the Pay-by-Phone integration module may beconfigured to receive information from the third-party Pay-by-Phone APIincluding, for example, a paid parking start date/time and a paidparking end date/time. In connection with step (5), the Pay-by-Phoneintegration module may be configured to compare a duration of paidparking indicated by the data received from step (3) with the actualparking duration determined by server system 140. If the actual parkingduration exceeds the paid parking duration, processing of step (6) isperformed, otherwise step (6) is not performed.

In an embodiment in which an API provided by server system 140 is usedfor integration, the Pay-by-Phone system will obtain only the vehicledetection information (actual vehicle parking start and end times) fromserver system 140 since typically vehicle license plate information isalready possessed by the Pay-by-Phone system. The end result is thePay-by-Phone system will have the precise information required togenerate a list of vehicle license plates with parking violationdetails.

Typically when a Pay-by-Phone system or its staff is agreeable to usingthe API provided by server system 140, the Pay-by-Phone system or itsstaff is prepared to undertake some degree of custom softwaredevelopment generally for the purpose of receiving some useful data fromserver system 140. In this particular case, the useful data is actualvehicle parking duration, indicating an actual amount of time aparticular vehicle has occupied a specific destination location, for aparticular vehicle license plate number or destination location number.In general, Pay-by-Phone systems already have the vehicle license platenumber or destination location number, as they are generally obtained aspart of the end user payment process.

Integration via a Pay-by-Phone API provided by server system 140 may beaccording to one of two options. First is a data inflow model, in whichthe third party parking payment system pushes information into serversystem 140. Second is a data outflow model, in which server system 140pushes all relevant information to the Pay-by-Phone system.

In an embodiment, server system 140 may include an inflow-basedPay-by-Phone API for use by a third-party parking payment system via,for example, network 110, in which the third party parking paymentsystem pushes information into server system 140 via the inflow-basedPay-by-Phone API. The inflow-based Pay-by-Phone API may be configured toperform steps including, for example: (1) waiting for the third partyparking payment system to establish a connection; (2) authenticating theidentity of the third party parking payment system; (3) receiving datafrom the third party parking payment system; (4) processing and storingdata to database 141; (5) determining parking violation status; (6)marking use of a destination location as a parking violation in database141 for further processing, for example, by the below-described parkingviolation pre-processing logic; (7) periodic confirmation that theinflow-based Pay-by-Phone API is active and valid, such as, but notlimited to, by use of “keep-alive”/“ping” messages or commands; and (8)closing the connection when requested or upon timeout. In connectionwith step (1), this may involve listening on a TCP or UDP port for thethird party parking payment system to establish the connection. Inconnection with step (2), the third party parking payment system isrequired to authenticate itself through the inflow-based Pay-by-PhoneAPI before data exchange is permitted. If authentication fails, a retrymechanism may be provided, and after several unsuccessful attempts theconnection may be dropped. In connection with step (3), server system140 may be configured to, via the inflow-based Pay-by-Phone API, receivevehicle parking information from the third party parking payment systemevery time a vehicle/driver pays for parking. The received informationfor a particular vehicle may include, for example, vehicle license plateinformation (including plate number and state/province information), anidentifier for a destination location for which payment was received inconnection with the particular vehicle (which may require translation toan identifier used internally by server system 140), a paid parkingstart date/time, and paid parking end date/time. In connection with step(5), a Pay-by-Phone integration module included in server system 140 maybe configured to compare a duration of paid parking indicated by thedata received from step (3) with the actual parking duration determinedby server system 140. If the actual parking duration exceeds the paidparking duration, processing of step (6) is performed, otherwise step(6) is not performed.

In an embodiment, server system 140 may include an outflow-basedPay-by-Phone API for use by a third-party parking payment system via,for example, network 110, in which server system 140 pushes all relevantinformation to the third-party Pay-by-Phone system. The outflow-basedPay-by-Phone API may be configured to perform steps including, forexample: (1) waiting for the third party parking payment system toestablish a connection; (2) authenticating the identity of the thirdparty parking payment system; (3) uploading data to the third partyparking payment system; (4) receiving an acknowledgement for theuploaded data; (5) responding to “keep-alive”/“ping” requests from thethird party parking payment system; and (6) closing the connection whenrequested or upon timeout. In connection with step (1), this may involvelistening on a TCP or UDP port for the third party parking paymentsystem to establish the connection. In connection with step (2), thethird party parking payment system is required to authenticate itselfthrough the outflow-based Pay-by-Phone API before data exchange ispermitted. If authentication fails, a retry mechanism may be provided,and after several unsuccessful attempts the connection may be dropped.In connection with step (3), server system 140 may be configured to, viathe outflow-based Pay-by-Phone API, upload vehicle parking informationto the third party parking payment system every time a vehicle vacates adestination location associated with the third party parking paymentsystem. The uploaded information for a particular vehicle may include,for example, vehicle license plate information (including plate numberand state/province information), an identifier for the destinationlocation recognized by the third party parking payment system (which mayrequire translation from an identifier used internally by server system140), an actual date/time the particular vehicle began use of thedestination location, and an actual date/time the particular vehiclevacated the destination location.

In another embodiment, integration between a Pay-by-Phone system andserver system 140 takes place in two steps. First, server system 140obtains vehicle payment information from the Pay-by-Phone system (suchas paid parking start and end times, parking space number, vehiclelicense plate number) by using the Pay-by-Phone system-provided API. Atthis stage, the server system 140 will have the precise informationrequired to generate a list of vehicle license plates with parkingviolation details. However, instead of server system 140 presenting thislist to a parking operator, there may be instances where an incumbentparking payment vendor wishes to retain control over how the parkingviolation information flows back to the parking operator. This can beaccomplished by sending the vehicle violation data back to thePay-by-Phone system by using the API provided by server system 140. ThePay-by-Phone system will then have total control over how the list of“offending” vehicles is presented to the parking operator (for example,by way of a custom software application developed by the Pay-by-Phonesystem vendor).

A significant difference between this integration method and onlyrelying on integration via the API provided by server system 140 is thatthis integration method requires minimal development effort on the partof the Pay-by-Phone system vendor. Since all the key information to bepresented to the parking operator is already supplied by the APIprovided by server system 140 without any further analysis required,this information can simply be displayed or forwarded to the parkingoperator with minimal software development. This integration approachrepresents a “pull” followed by a “push” model.

The premise of Pay-by-Space is that once an end user parks a vehicle,the end user notes the space number of the parking spot. When the enduser pays for parking at a Pay-by-Space machine, the end user enters astall number followed by parking duration desired. Once the payment iscomplete, the end user does not need to leave a ticket in the vehicle,instead the end user can simply continue on to wherever they are going.Pay-by-Space works equally well for both on-street curbside parking andoff-street parking lots/parking garages.

Pay-by-Space is increasingly becoming popular compared toPay-and-Display, which has a longer history of operation. Parkingoperations using Pay-by-Space are generally more efficient since parkingenforcement personnel no longer have to walk to each vehicle and leaninto the windshield in order to manually check a printed expirationtime. Instead, they can drive in a patrol vehicle and perform visualchecks from a distance to see whether an occupied parking space has beenpaid for. In addition, Pay-by-Space offers enhanced a user experiencesince end users no longer have to walk back to their vehicles to displaythe ticket after paying at the machine—instead, they can continue on totheir destination.

Similar to the above-described Pay-by-Phone, integration can beaccomplished by using either a Pay-by-Space system-provided API or anAPI provided by server system 140. The objective is for either system togain access to all the information required to generate a list ofvehicle license plates with parking violation information.

In an embodiment in which server system 140 is integrated via aPay-by-Space system-provided API, server system 140 obtains vehiclepayment information from the Pay-by-Space system (such as paid parkingstart and end times, destination location number, and vehicle licenseplate number). The end result is server system 140 will have the preciseinformation required to generate a list of vehicle license plates withparking violation details.

In an embodiment, server system 140 may include a Pay-by-Spaceintegration module, which may be a software component that extractsparking payment information from a third party parking payment systemwhich server system 140 uses for parking self-enforcement. Theintegration module for pay-by-space performs steps including: (1)establishing a connection with a third-party Pay-by-Space API; (2)issuing a request for data; (3) receiving data; (4) processing andstoring data to database 141; (5) determining parking violation status;(6) marking use of a destination location as a parking violation indatabase 141 for further processing, for example, by the below-describedparking violation pre-processing logic; (7) periodic confirmation thatthe third-party Pay-by-Space API is active and valid, such as, but notlimited to, by use of “keep-alive”/“ping” messages or commands; and (8)repeating steps 2-7 or closing the connection. In connection with step(1), an API typically specifies the mechanism through which a connectionto it can be established. The Pay-by-Space integration module isconfigured to observe the protocol required by the third-partyPay-by-Space API and establishes a connection.

In connection with step (2), when server system 140 determines that aparticular vehicle has vacated a destination location, server system 140signals this event to the Pay-by-Space integration module, such as via adatabase or Inter-Process Communication (IPC) call, in response to whichthe Pay-by-Space integration module issues a request to the third-partyPay-by-Space API for parking payment information for the particularvehicle. Data sent to the third-party Pay-by-Space API may include, forexample, an identifier for the destination location recognized by thePay-by-Space API (which may require translation from an identifier usedinternally by server system 140), and a date/time the particular vehiclevacated the destination location.

In connection with step (3), the Pay-by-Space integration module may beconfigured to receive information from the third-party Pay-by-Space APIincluding, for example, a paid parking start date/time and a paidparking end date/time. In connection with step (5), the Pay-by-Spaceintegration module may be configured to compare a duration of paidparking indicated by the data received from step (3) with the actualparking duration determined by server system 140. If the actual parkingduration exceeds the paid parking duration, processing of step (6) isperformed, otherwise step (6) is not performed.

In an embodiment in which server system 140 is integrated with aPay-by-Space system via an API provided by server system 140, serversystem 140 transmits vehicle information to the Pay-by-Space system(such as vehicle license plate number, destination location, and actualparking start and end times). The end result is the Pay-by-Space systemwill have the precise information required to generate a list of vehiclelicense plates with parking violation details.

Typically when a Pay-by-Space system or its staff is agreeable to usingthe API provided by server system 140, the Pay-by-Space system or itsstaff is prepared to undertake some degree of custom softwaredevelopment generally for the purpose of receiving some useful data fromserver system 140. In this particular case, the useful data is actualvehicle parking duration, an actual amount of time a particular vehiclehas occupied a specific destination location, for a particular vehiclelicense plate number or space number. In general, Pay-by-Space systemsalready have the vehicle license plate number or space number, as thismay be obtained as part of the end user payment process.

Integration via a Pay-by-Space API provided by server system 140 may beaccording to one of two options. First is a data inflow model, in whichthe third party parking payment system pushes information into serversystem 140. Second is a data outflow model, in which server system 140pushes all relevant information to the third-party Pay-by-Space system.

In an embodiment, server system 140 may include an inflow-basedPay-by-Space API for use by a third-party parking payment system via,for example, network 110, in which the third party parking paymentsystem pushes information into server system 140 via the inflow-basedPay-by-Space API. The inflow-based Pay-by-Space API may be configured toperform steps including, for example: (1) waiting for the third partyparking payment system to establish a connection; (2) authenticating theidentity of the third party parking payment system; (3) receiving datafrom the third party parking payment system; (4) processing and storingdata to database 141; (5) determining parking violation status; (6)marking use of a destination location as a parking violation in database141 for further processing, for example, by the below-described parkingviolation pre-processing logic; (7) periodic confirmation that theinflow-based Pay-by-Space API is active and valid, such as, but notlimited to, by use of “keep-alive”/“ping” messages or commands; and (8)closing the connection when requested or upon timeout. In connectionwith step (1), this may involve listening on a TCP or UDP port for thethird party parking payment system to establish the connection. Inconnection with step (2), the third party parking payment system isrequired to authenticate itself through the inflow-based Pay-by-SpaceAPI before data exchange is permitted. If authentication fails, a retrymechanism may be provided, and after several unsuccessful attempts theconnection may be dropped. In connection with step (3), server system140 may be configured to, via the inflow-based Pay-by-Space API, receivevehicle parking information from the third party parking payment systemevery time a vehicle/driver pays for parking. The received informationfor a particular vehicle may include, for example, an identifier for adestination location for which payment was received in connection withthe particular vehicle (which may require translation to an identifierused internally by server system 140), a paid parking start date/time,and paid parking end date/time. In connection with step (5), aPay-by-Space integration module included in server system 140 may beconfigured to compare a duration of paid parking indicated by the datareceived from step (3) with the actual parking duration determined byserver system 140. If the actual parking duration exceeds the paidparking duration, processing of step (6) is performed, otherwise step(6) is not performed.

In an embodiment, server system 140 may include an outflow-basedPay-by-Space API for use by a third-party parking payment system via,for example, network 110, in which server system 140 pushes all relevantinformation to the third-party Pay-by-Space system. The outflow-basedPay-by-Space API may be configured to perform steps including, forexample: (1) waiting for the third party parking payment system toestablish a connection; (2) authenticating the identity of the thirdparty parking payment system; (3) uploading data to the third partyparking payment system; (4) receiving an acknowledgement for theuploaded data; (5) responding to “keep-alive”/“ping” requests from thethird party parking payment system; and (6) closing the connection whenrequested or upon timeout. In connection with step (1), this may involvelistening on a TCP or UDP port for the third party parking paymentsystem to establish the connection. In connection with step (2), thethird party parking payment system is required to authenticate itselfthrough the outflow-based Pay-by-Space API before data exchange ispermitted. If authentication fails, a retry mechanism may be provided,and after several unsuccessful attempts the connection may be dropped.In connection with step (3), server system 140 may be configured to, viathe outflow-based Pay-by-Space API, upload vehicle parking informationto the third party parking payment system every time a vehicle vacates adestination location associated with the third party parking paymentsystem. The uploaded information for a particular vehicle may include,for example, vehicle license plate information (including plate numberand state/province information), an identifier for the destinationlocation recognized by the third party parking payment system (which mayrequire translation from an identifier used internally by server system140), an actual date/time the particular vehicle began use of thedestination location, and an actual date/time the particular vehiclevacated the destination location.

In another embodiment, server system 140 is configured to integrate witha Pay-by-Space system in two steps. First, server system 140 obtainsvehicle payment information from the Pay-by-Space system (for example,paid parking start and end times, parking space number, vehicle licenseplate number) by using a Pay-by-Space system-provided API. At thisstage, server system 140 will have the precise information required togenerate a list of vehicle license plates with parking violationdetails. However, instead of server system 140 presenting this list tothe parking operator, there may be instances where an incumbent parkingpayment vendor wishes to retain control over how the parking violationinformation flows back to the parking operator. This can be accomplishedby sending the vehicle violation data back to the Pay-by-Space system byusing the API provided by server system 140. The Pay-by-Space systemwill then have total control over how the list of “offending” vehiclesis presented to the parking operator (such as by way of a customsoftware application developed by the Pay-by-Space system vendor).

A significant difference between this integration method and integrationwith the Pay-by-Space system via the API provided by server system 140is that this integration method requires minimal development effort onthe part of the Pay-by-Space system vendor. Since all the keyinformation to be presented to the parking operator is already suppliedby the API provided by server system 140 without any further analysisrequired, this information can simply be displayed or forwarded to theparking operator with minimal software development. This integrationapproach represents a “pull” followed by a “push” model.

The premise of a Pay-and-Display is that once an end user pays forparking, the end user needs to leave a ticket inside the vehicle on thedashboard in order for parking enforcement personnel to perform visualchecks of the ticket to ensure that the period of time paid for by theend user has not expired.

Historically Pay-and-Display dominated off-street parking lots andparking garages. Despite the recent popularity of Pay-by-Space machines,Pay-and-Display still has a vast installed base. There are alsoPay-and-Display systems deployed for curbside parking.

In order to provide self-enforcement capability, server system 140requires the following information: vehicle license plate information(including state/province), actual parking start and end times, paidparking duration.

In an embodiment integrating server system 140 with a Pay-and-Displaysystem, server system 140 gathers vehicle license plate information viathe identification cameras, while the destination cameras provide actualparking start and end times by allowing server system 140 to determinewhen a vehicle pulls into a destination location and when the vehicleleaves. Paid parking duration may be determined by utilizing destinationcameras with pan, tilt, and zoom capability, and zooming in to extract abar code or printed characters from the display ticket itself which anend user places on a vehicle dashboard.

In an embodiment, to facilitate the reading of such information, aPay-and-Display machine may be configured to use a larger font for theprinted characters or print larger versions of other identifyingfeatures. In another embodiment, a Pay-and-Display machine may beconfigured to ask for a license plate number as part of a paymentprocess, whereas a conventional system would only ask for an amount oftime a driver wishes to park a vehicle. In an embodiment, aPay-and-Display machine may utilize rolls of display ticket paper withan RFID chip embedded within each display ticket.

In an embodiment, server system 140 may be configured to performpedestrian tracking based on multiple images captured via one or morecameras, such as a camera including a Pay-and-Display machine within itsfield of view. With pedestrian tracking, server system 140 may beconfigured to track a driver and/or passenger of a vehicle as they walkto and/or from a vehicle for which payment is obtained via thePay-and-Display machine, much as server system 140 performs vehicletracking. Based on the tracked pedestrian movement, the payment may belinked to a particular location for a vehicle, and a correspondingvehicle identifier, such as a license plate number, determined by serversystem 140. Such an embodiment eliminates a need to capture informationdirectly from a printed ticket. In an example, server system 140 mayreceive a plurality of pedestrian images captured by one or morecameras, including one or more images demonstrating an individual was apassenger of the vehicle, and one or more images demonstrating thepassenger made a payment at a Pay-and-Display payment station for use ofa destination location. Server system 140 may be configured to performtracking by determining whether characteristics determined forpedestrians captured in the pedestrian images correspond over time,demonstrating that the passenger traveled between the vehicle and thepayment station, and made the payment. Based on this, server system 140may associate the payment with use of a particular destination locationby the vehicle.

With the above information available, server system 140 is able toproduce a list of vehicle license plates with detailed violationinformation. This holds true regardless of whether the Pay-and-Displaysystem supports real-time wireless networking since the Pay-and-Displaysystem itself does not possess any of the said information.

In an embodiment integrating server system 140 with a Pay-and-Displaysystem via an API provided by server system 140, to provideself-enforcement capability, server system 140 requires vehicle licenseplate information (including state/province information), actual parkingstart and end times, and paid parking duration.

Server system 140 gathers vehicle license plate information via theidentification cameras, while the destination cameras allow serversystem 140 to determine actual parking start and end times by detectingwhen a vehicle pulls into a parking space and when it leaves. Paidparking duration is determined by using destination cameras with pan,tilt, and zoom functionality, enabling zooming in to extracting the barcode or printed characters from the display ticket itself which endusers place on the vehicle dash.

The Pay-and-Display API provided by server system 140 may be configuredto perform steps including, for example: (1) waiting for the third partyparking payment system to establish a connection; (2) authenticating theidentity of the third party parking payment system; (3) uploading datato the third party parking payment system; (4) receiving anacknowledgement for the uploaded data; (5) responding to“keep-alive”/“ping” requests from the third party parking paymentsystem; and (6) closing the connection when requested or upon timeout.In connection with step (1), this may involve listening on a TCP or UDPport for the third party parking payment system to establish theconnection. In connection with step (2), the third party parking paymentsystem is required to authenticate itself through the Pay-and-DisplayAPI before data exchange is permitted. If authentication fails, a retrymechanism may be provided, and after several unsuccessful attempts theconnection may be dropped. In connection with step (3), server system140 may be configured to, via the Pay-and-Display API, upload vehicleparking information to the third party parking payment system every timea vehicle vacates a destination location associated with the third partyparking payment system. The uploaded information for a particularvehicle may include, for example, vehicle license plate information(including plate number and state/province information), an identifierfor the destination location recognized by the third party parkingpayment system (which may require translation from an identifier usedinternally by server system 140), an actual date/time the particularvehicle began use of the destination location, and an actual date/timethe particular vehicle vacated the destination location.

When server system 140 is configured to operate in this mode, serversystem 140 transmits vehicle parking violation information to thePay-and-Display system (such as vehicle license plate number, paidparking start and end times, actual parking start and end times). Theend result is the third-party Pay-and-Display system will have theprecise information required to generate a list of vehicle licenseplates with parking violation details.

Typically when a Pay-and-Display system or its staff is agreeable tousing the API provided by server system 140, the Pay-and-Display systemor its staff is prepared to undertake some degree of custom softwaredevelopment generally for the purpose of receiving some useful data fromserver system 140. In this particular case, the useful data is the paidparking start and end times, and actual vehicle parking start and endtimes for a particular vehicle license plate number. In general,Pay-and-Display systems are paper-based (as they rely on a printeddisplay ticket) and do not possess any information with respect tovehicle identity or destination location number. While Pay-and-Displaysystems do possess paid parking start time and paid parking duration,these systems have no knowledge of which vehicle the paid parkingcorrelates to. The Pay-and-Display system has the option of presenting alist of vehicles with parking violation information to the parkingoperator by using an API provided by server system 140. Through the API,the Pay-and-Display system can obtain the pertinent information fromserver system 140 and present it in any desired format.

As such, integration via the API provided by server system 140 is a“push” model where server system 140 “pushes” all relevant informationto the Pay-and-Display system. Subsequently, the Pay-and-Display systemdisplays a list of vehicle license plates with unpaid or expiredparking. In other words, Pay-and-Display integration via the APIprovided by server system 140 is generally not accomplished in a “pull”model.

In an embodiment, a Pay-and-Display machine with QR codes may beprovided. QR codes have become popular, and can easily hold all of therelevant information typically contained on a Pay-and-Display ticket(such as parking duration, parking expiration time, amount paid, date,and time).

In an embodiment, a Pay-by-Phone system may overlay a Pay-and-Displayparking system, and server system 140 may be integrated with thePay-by-Phone payment system via API. In such a case, where aPay-by-Phone system-provided API is used, server system 140 willtransmit vehicle detection information to the Pay-by-Phone system sincetypically the vehicle license plate information is already possessed bythe Pay-by-Phone system. The end result is the Pay-by-Phone system willhave the information required to generate a list of vehicle licenseplates with parking violation details.

While traditional curbside parking meters (single or double space) wouldonly accept coins with no built-in intelligence or networkingcapability, Pay-by-Phone has become a popular “overlay” payment approachwhich works well with traditional curbside parking meters. Recentlythere has also been an emergence of curbside parking meters which acceptboth coins and credit cards, and have networking capability built-in.

Traditional parking meters generally have either a mechanical or an LCDdisplay “expired flag” to indicate that a paid parking period hasexpired or parking is simply not paid. When parking has been paid andthere is time remaining, the mechanical expired flag (typically red) ishidden from view, whereas the LCD display shows a clear display on theside of the meter facing the roadway. In contrast, when paid parking hasexpired or when no payment has been made, the red mechanical expiredflag is prominently displayed in the “up” position, while the LCDdisplay facing the roadway flashes alternately between a clear displayand a dark display (sometimes also with a flashing red LED). Both ofthese are generally referred to as an expired flag.

In an embodiment, server system 140 can interface with traditionalcurbside parking meters using a combination of information gathered fromboth identification and destination cameras. In addition to identifyingvehicle parking status (such as, but not limited to, parking start andend times), images captured by the identification and destinationcameras can pinpoint paid parking status (such as, but not limited to,whether a prepaid time has expired) on the parking meters.

Server system 140 gathers vehicle license plate information via theimages captured by identification cameras, while images captured bydestination cameras enable server system 140 to determine actual parkingstart and end times by detecting when a vehicle pulls into a parkingspace and when it leaves. Time of expiration of paid parking isdetermined by zooming in with both location and destination cameras andmonitoring the visible status of the expiration flag on each parkingmeter. Based on this information, server system 140 may determineparking violation status, and mark use of a destination location as aparking violation in database 141 for further processing, for example,by the below-described parking violation pre-processing logic. Insituations where an expiration flag is not readable by server system140, server system 140 may be configured to indicate an exception to aparking operator or staff and/or make use of a Live Operator (discussedbelow) to manually determine parking violation status and/or exceptionflag status.

A destination image captured by a destination camera from across thestreet will have sufficient angle of view to monitor the status of theexpiration flag on each parking meter. Additionally, an identificationcamera with pan, tilt, and zoom capability can be provided withsufficient zoom power and angle of view to also monitor the expirationflag status on each parking meter. With the above information available,server system 140 is able to produce a list of vehicle license plateswith detailed violation information.

In an embodiment integrating server system 140 with a TraditionalCurbside Parking Meter system via an API provided by server system 140,to provide self-enforcement capability, server system 140 requiresvehicle license plate information (including state/provinceinformation), actual parking start and end times, and paid parkingduration.

Server system 140 gathers vehicle license plate information via theimages captured by identification cameras, while images captured bydestination cameras enable server system 140 to determine actual parkingstart and end times by detecting when a vehicle pulls into a parkingspace and when it leaves. Time of expiration of paid parking isdetermined by zooming in with both location and destination cameras andmonitoring the visible status of the expiration flag on each parkingmeter. Based on this information, server system 140 may determineparking violation status, and mark use of a destination location as aparking violation in database 141 for further processing, for example,by the below-described parking violation pre-processing logic. Insituations where an expiration flag is not readable by server system140, server system 140 may be configured to indicate an exception to aparking operator or staff and/or make use of a Live Operator (discussedbelow) to manually determine parking violation status and/or exceptionflag status.

The Traditional Curbside Parking Meter API provided by server system 140may be configured to perform steps including, for example: (1) waitingfor the third party parking payment system to establish a connection;(2) authenticating the identity of the third party parking paymentsystem; (3) uploading data to the third party parking payment system;(4) receiving an acknowledgement for the uploaded data; (5) respondingto “keep-alive”/“ping” requests from the third party parking paymentsystem; and (6) closing the connection when requested or upon timeout.In connection with step (1), this may involve listening on a TCP or UDPport for the third party parking payment system to establish theconnection. In connection with step (2), the third party parking paymentsystem is required to authenticate itself through the TraditionalCurbside Parking Meter API before data exchange is permitted. Ifauthentication fails, a retry mechanism may be provided, and afterseveral unsuccessful attempts the connection may be dropped. Inconnection with step (3), server system 140 may be configured to, viathe Traditional Curbside Parking Meter API, upload vehicle parkinginformation to the third party parking payment system every time avehicle vacates a destination location associated with the third partyparking payment system. The uploaded information for a particularvehicle may include, for example, vehicle license plate information(including plate number and state/province information), an identifierfor the destination location recognized by the third party parkingpayment system (which may require translation from an identifier usedinternally by server system 140), an actual date/time the particularvehicle began use of the destination location, and an actual date/timethe particular vehicle vacated the destination location.

When server system 140 is configured to operate in this mode, serversystem 140 transmits vehicle parking violation information to theTraditional Curbside Parking Meter system, which is typically a paymentcollection/tallying system with no connectivity to the actual curbsidemeters. The end result is that the Traditional Curbside Parking Metersystem will have the precise information required to generate a list ofvehicle license plates with parking violation details.

Typically when a Traditional Curbside Parking Meter system or its staffis agreeable to using the API provided by server system 140, theTraditional Curbside Parking Meter system or its staff is prepared toundertake some degree of custom software development generally for thepurpose of receiving some useful data from server system 140. In thisparticular case, the useful data is the expired or unpaid parkingstatus, and actual vehicle parking start and end times for a particularvehicle license plate number. In general, Traditional Curbside ParkingMeter systems do not possess any information with respect to vehicleidentity or space number.

As such, integration via the API provided by server system 140 is a“push” model where server system 140 “pushes” all relevant informationto the Traditional Curbside Parking Meter system. Subsequently, theTraditional Curbside Parking Meter system displays a list of vehiclelicense plates with unpaid or expired parking.

In an embodiment, a Pay-by-Phone system may overlay a traditionalcurbside parking system, and server system 140 may be integrated withthe Pay-by-Phone payment system via API. In such a case, where aPay-by-Phone system-provided API is used, server system 140 willtransmit vehicle detection information to the Pay-by-Phone system sincetypically the vehicle license plate information is already possessed bythe Pay-by-Phone system. The end result is the Pay-by-Phone system willhave the information required to generate a list of vehicle licenseplates with parking violation details.

More recently, curbside parking meters (single or double space) havebeen introduced which perform credit card authorization using a built-in3G or WiFi modem. In an embodiment, server system 140 can be configuredto interface with this type of parking meters through a Curbside ParkingMeter API. Specifically, since these curbside parking meters alreadyhave built-in wireless network connectivity and an associated backendserver, it is not difficult for server system 140 to integrate with thisthird-party backend server.

When server system 140 is configured to operate in this mode, anintegration module for the Curbside Parking Meter API performs stepsincluding: (1) establishing a connection with a third-party CurbsideParking Meter API; (2) issuing a request for data; (3) receiving data;(4) processing and storing data to database 141; (5) determining parkingviolation status; (6) marking use of a destination location as a parkingviolation in database 141 for further processing, for example, by thebelow-described parking violation pre-processing logic; (7) periodicconfirmation that the Curbside Parking Meter API is active and valid,such as, but not limited to, by use of “keep-alive”/“ping” messages orcommands; and (8) repeating steps 2-7 or closing the connection. Inconnection with step (1), an API typically specifies the mechanismthrough which a connection to it can be established. The integrationmodule for the Curbside Parking Meter API is configured to observe theprotocol required by the Curbside Parking Meter API and establishes aconnection.

In connection with step (2), when server system 140 determines that aparticular vehicle has vacated a destination location, server system 140signals this event to the integration module for the Curbside ParkingMeter API, such as via a database or Inter-Process Communication (IPC)call, in response to which the integration module for the CurbsideParking Meter API issues a request to the third-party Curbside ParkingMeter API for parking payment information for the particular vehicle.Data sent to the third-party Curbside Parking Meter API may include, forexample, an identifier for the destination location recognized by theCurbside Parking Meter API (which may require translation from anidentifier used internally by server system 140), and a date/time theparticular vehicle vacated the destination location.

In connection with step (3), the integration module for the CurbsideParking Meter API may be configured to receive information from thethird-party Curbside Parking Meter API including, for example, anidentifier for the destination location recognized by the CurbsideParking Meter API (which may require translation to an identifier usedinternally by server system 140), a paid parking start date/time, and apaid parking end date/time. In connection with step (5), the integrationmodule for the Curbside Parking Meter API may be configured to compare aduration of paid parking indicated by the data received from step (3)with the actual parking duration determined by server system 140. If theactual parking duration exceeds the paid parking duration, processing ofstep (6) is performed, otherwise step (6) is not performed.

As such, integration via Curbside Parking Meter API is a “pull” modelwhere server system 140 “pulls” all relevant information the third-partyCurbside Parking Meter system is capable of outputting, and addsautomated vehicle violation information determined by server system 140to produce a list of vehicle license plates with unpaid or expiredparking. In other words, integration via Curbside Parking Meter API isgenerally not accomplished in a “push” model.

In an embodiment in which server system 140 is configured to integratewith a Curbside Parking Meter system via an API provided by serversystem 140, server system 140 transmits vehicle information to theCurbside Parking Meter system (such as vehicle license plate number,destination location, and actual parking start and end times). The endresult is the Curbside Parking Meter system will have the preciseinformation required to generate a list of vehicle license plates withparking violation details.

Typically when a Curbside Parking Meter system or its staff is agreeableto using the API provided by server system 140, the Curbside ParkingMeter system or its staff is prepared to undertake some degree of customsoftware development generally for the purpose of receiving some usefuldata from server system 140. In this particular case, the useful data isactual vehicle parking duration, the actual amount of time a particularvehicle has occupied a specific destination location, for a particularvehicle license plate number or destination location number. In general,Curbside Parking Meter systems already have the vehicle license platenumber or space number, as this information may be obtained as part ofthe end user payment process.

Integration via a Curbside Parking Meter API provided by server system140 may be according to one of two options. First is a data inflowmodel, in which the third party parking payment system pushesinformation into server system 140. Second is a data outflow model, inwhich server system 140 pushes all relevant information to the CurbsideParking Meter system.

In an embodiment, server system 140 may include an inflow-based CurbsideParking Meter API for use by a third-party parking payment system via,for example, network 110, in which the third party parking paymentsystem pushes information into server system 140 via the inflow-basedCurbside Parking Meter API. The inflow-based Curbside Parking Meter APImay be configured to perform steps including, for example: (1) waitingfor the third party parking payment system to establish a connection;(2) authenticating the identity of the third party parking paymentsystem; (3) receiving data from the third party parking payment system;(4) processing and storing data to database 141; (5) determining parkingviolation status; (6) marking use of a destination location as a parkingviolation in database 141 for further processing, for example, by thebelow-described parking violation pre-processing logic; (7) periodicconfirmation that the inflow-based Curbside Parking Meter API is activeand valid, such as, but not limited to, by use of “keep-alive”/“ping”messages or commands; and (8) closing the connection when requested orupon timeout. In connection with step (1), this may involve listening ona TCP or UDP port for the third party parking payment system toestablish the connection. In connection with step (2), the third partyparking payment system is required to authenticate itself through theinflow-based Curbside Parking Meter API before data exchange ispermitted. If authentication fails, a retry mechanism may be provided,and after several unsuccessful attempts the connection may be dropped.In connection with step (3), server system 140 may be configured to, viathe inflow-based Curbside Parking Meter API, receive vehicle parkinginformation from the third party parking payment system every time avehicle/driver pays for parking. The received information for aparticular vehicle may include, for example, an identifier for adestination location for which payment was received in connection withthe particular vehicle (which may require translation to an identifierused internally by server system 140), a paid parking start date/time,and a paid parking end date/time. In connection with step (5), aCurbside Parking Meter integration module included in server system 140may be configured to compare a duration of paid parking indicated by thedata received from step (3) with the actual parking duration determinedby server system 140. If the actual parking duration exceeds the paidparking duration, processing of step (6) is performed, otherwise step(6) is not performed.

In an embodiment, server system 140 may include an outflow-basedCurbside Parking Meter API for use by a third-party parking paymentsystem via, for example, network 110, in which server system 140 pushesall relevant information to the third-party Curbside Parking Metersystem. The outflow-based Curbside Parking Meter API may be configuredto perform steps including, for example: (1) waiting for the third partyparking payment system to establish a connection; (2) authenticating theidentity of the third party parking payment system; (3) uploading datato the third party parking payment system; (4) receiving anacknowledgement for the uploaded data; (5) responding to“keep-alive”/“ping” requests from the third party parking paymentsystem; and (6) closing the connection when requested or upon timeout.In connection with step (1), this may involve listening on a TCP or UDPport for the third party parking payment system to establish theconnection. In connection with step (2), the third party parking paymentsystem is required to authenticate itself through the outflow-basedCurbside Parking Meter API before data exchange is permitted. Ifauthentication fails, a retry mechanism may be provided, and afterseveral unsuccessful attempts the connection may be dropped. Inconnection with step (3), server system 140 may be configured to, viathe outflow-based Curbside Parking Meter API, upload vehicle parkinginformation to the third party parking payment system every time avehicle vacates a destination location associated with the third partyparking payment system. The uploaded information for a particularvehicle may include, for example, vehicle license plate information(including plate number and state/province information), an identifierfor the destination location recognized by the third party parkingpayment system (which may require translation from an identifier usedinternally by server system 140), an actual date/time the particularvehicle began use of the destination location, and an actual date/timethe particular vehicle vacated the destination location.

In an embodiment, server system 140 may be configured to integrate witha Curbside Parking Meter system in two steps. First, server system 140obtains vehicle payment information from the Curbside Parking Metersystem (such as paid parking start and end times, parking space number,vehicle license plate number) by using the Curbside Parking Metersystem-provided API. At this stage, server system 140 will have theprecise information required to generate a list of vehicle licenseplates with parking violation details. However, instead of server system140 presenting this list to a parking operator, there may be instanceswhere an incumbent parking payment vendor wishes to retain control overhow the parking violation information flows back to the parkingoperator. This can be accomplished by sending the vehicle violation databack to the Curbside Parking Meter system by using an API provided byserver system 140. The Curbside Parking Meter system will then havetotal control over how the list of “offending” vehicles is presented tothe parking operator (such as by use of a custom software applicationdeveloped by the Curbside Parking Meter system vendor).

A significant difference between this integration method and integrationvia the API provided by server system 140 is that this integrationmethod requires minimal development effort on the part of the CurbsideParking Meter system vendor. Since all the key information to bepresented to the parking operator is already supplied by the APIprovided by server system 140 without any further analysis required,this information can simply be displayed or forwarded to the parkingoperator with minimal software development. This integration approachrepresents a “pull” followed by a “push” model.

Similar to Pay-by-Space where a parking payment is associated with aparticular numbered parking space (in which a vehicle parks),Pay-by-Plate associates a parking payment to a vehicle with a particularlicense plate parked in a parking space. Manual enforcement ofPay-by-Plate parking typically involves the enforcement officer drivingpast the physical parking space to ensure license plates of all parkedvehicles are on a list of license plates with paid status. Conversely,any parked vehicles with license plates not on the “paid” list wouldreceive a parking citation.

Similar to the above-described Pay-by-Phone and Pay-by-Space,integration can be accomplished by using either a Pay-by-Platesystem-provided API or an API provided by server system 140. Theobjective is for either system to gain access to all the informationrequired to generate a list of vehicle license plates with parkingviolation information.

In an embodiment in which server system 140 is integrated via aPay-by-Plate system-provided API, server system 140 obtains vehiclepayment information from the Pay-by-Plate system (such as paid parkingstart and end times and vehicle license plate numbers). The end resultis server system 140 will have the precise information required togenerate a list of vehicle license plates with parking violationdetails.

In an embodiment, server system 140 may include a Pay-by-Plateintegration module, which may be a software component that extractsparking payment information from a third party parking payment systemwhich server system 140 uses for parking self-enforcement. Theintegration module for pay-by-plate performs steps including: (1)establishing a connection with a third-party Pay-by-Plate API; (2)issuing a request for data; (3) receiving data; (4) processing andstoring data to database 141; (5) determining parking violation status;(6) marking use of a destination location as a parking violation indatabase 141 for further processing, for example, by the below-describedparking violation pre-processing logic; (7) periodic confirmation thatthe third-party Pay-by-Plate API is active and valid, such as, but notlimited to, by use of “keep-alive”/“ping” messages or commands; and (8)repeating steps 2-7 or closing the connection. In connection with step(1), an API typically specifies the mechanism through which a connectionto it can be established. The Pay-by-Plate integration module isconfigured to observe the protocol required by the third-partyPay-by-Plate API and establishes a connection.

In connection with step (2), when server system 140 determines that aparticular vehicle has vacated a destination location, server system 140signals this event to the Pay-by-Plate integration module, such as via adatabase or Inter-Process Communication (IPC) call, in response to whichthe Pay-by-Plate integration module issues a request to the third-partyPay-by-Plate API for parking payment information for the particularvehicle. Data sent to the third-party Pay-by-Plate API may include, forexample, vehicle license plate information (including plate number andstate/province information), and a date/time the particular vehiclevacated the destination location.

In connection with step (3), the Pay-by-Plate integration module may beconfigured to receive information from the third-party Pay-by-Plate APIincluding, for example, a paid parking start date/time and a paidparking end date/time. In connection with step (5), the Pay-by-Plateintegration module may be configured to compare a duration of paidparking indicated by the data received from step (3) with the actualparking duration determined by server system 140. If the actual parkingduration exceeds the paid parking duration, processing of step (6) isperformed, otherwise step (6) is not performed.

In an embodiment in which server system 140 is integrated with aPay-by-Plate system via an API provided by server system 140, serversystem 140 transmits vehicle information to the Pay-by-Plate system(such as vehicle license plate number, destination location, and actualparking start and end times). The end result is the Pay-by-Plate systemwill have the precise information required to generate a list of vehiclelicense plates with parking violation details.

Typically when a Pay-by-Plate system or its staff is agreeable to usingthe API provided by server system 140, the Pay-by-Plate system or itsstaff is prepared to undertake some degree of custom softwaredevelopment generally for the purpose of receiving some useful data fromserver system 140. In this particular case, the useful data is actualvehicle parking duration, an actual amount of time a particular vehiclehas occupied a destination location, for a particular vehicle licenseplate number. By definition, Pay-by-Plate systems already have thevehicle license plate number, as this is obtained as part of the enduser payment process.

Integration via a Pay-by-Plate API provided by server system 140 may beaccording to one of two options. First is a data inflow model, in whichthe third party parking payment system pushes information into serversystem 140. Second is a data outflow model, in which server system 140pushes all relevant information to the third-party Pay-by-Plate system.

In an embodiment, server system 140 may include an inflow-basedPay-by-Plate API for use by a third-party parking payment system via,for example, network 110, in which the third party parking paymentsystem pushes information into server system 140 via the inflow-basedPay-by-Plate API. The inflow-based Pay-by-Plate API may be configured toperform steps including, for example: (1) waiting for the third partyparking payment system to establish a connection; (2) authenticating theidentity of the third party parking payment system; (3) receiving datafrom the third party parking payment system; (4) processing and storingdata to database 141; (5) determining parking violation status; (6)marking use of a destination location as a parking violation in database141 for further processing, for example, by the below-described parkingviolation pre-processing logic; (7) periodic confirmation that theinflow-based Pay-by-Plate API is active and valid, such as, but notlimited to, by use of “keep-alive”/“ping” messages or commands; and (8)closing the connection when requested or upon timeout. In connectionwith step (1), this may involve listening on a TCP or UDP port for thethird party parking payment system to establish the connection. Inconnection with step (2), the third party parking payment system isrequired to authenticate itself through the inflow-based Pay-by-PlateAPI before data exchange is permitted. If authentication fails, a retrymechanism may be provided, and after several unsuccessful attempts theconnection may be dropped. In connection with step (3), server system140 may be configured to, via the inflow-based Pay-by-Plate API, receivevehicle parking information from the third party parking payment systemevery time a vehicle/driver pays for parking. The received informationfor a particular vehicle may include, for example, vehicle license plateinformation (including plate number and state/province information), apaid parking start date/time, and paid parking end date/time. Inconnection with step (5), a Pay-by-Plate integration module included inserver system 140 may be configured to compare a duration of paidparking indicated by the data received from step (3) with the actualparking duration determined by server system 140. If the actual parkingduration exceeds the paid parking duration, processing of step (6) isperformed, otherwise step (6) is not performed.

In an embodiment, server system 140 may include an outflow-basedPay-by-Plate API for use by a third-party parking payment system via,for example, network 110, in which server system 140 pushes all relevantinformation to the third-party Pay-by-Plate system. The outflow-basedPay-by-Plate API may be configured to perform steps including, forexample: (1) waiting for the third party parking payment system toestablish a connection; (2) authenticating the identity of the thirdparty parking payment system; (3) uploading data to the third partyparking payment system; (4) receiving an acknowledgement for theuploaded data; (5) responding to “keep-alive”/“ping” requests from thethird party parking payment system; and (6) closing the connection whenrequested or upon timeout. In connection with step (1), this may involvelistening on a TCP or UDP port for the third party parking paymentsystem to establish the connection. In connection with step (2), thethird party parking payment system is required to authenticate itselfthrough the outflow-based Pay-by-Plate API before data exchange ispermitted. If authentication fails, a retry mechanism may be provided,and after several unsuccessful attempts the connection may be dropped.In connection with step (3), server system 140 may be configured to, viathe outflow-based Pay-by-Plate API, upload vehicle parking informationto the third party parking payment system every time a vehicle vacates adestination location associated with the third party parking paymentsystem. The uploaded information for a particular vehicle may include,for example, vehicle license plate information (including plate numberand state/province information), an identifier for the destinationlocation recognized by the third party parking payment system (which mayrequire translation from an identifier used internally by server system140), an actual date/time the particular vehicle began use of thedestination location, and an actual date/time the particular vehiclevacated the destination location.

In another embodiment, server system 140 is configured to integrate witha Pay-by-Plate system in two steps. First, server system 140 obtainsvehicle payment information from the Pay-by-Plate system (for example,paid parking start and end times, and vehicle license plate number) byusing a Pay-by-Plate system-provided API. At this stage, server system140 will have the precise information required to generate a list ofvehicle license plates with parking violation details. However, insteadof server system 140 presenting this list to the parking operator, theremay be instances where an incumbent parking payment vendor wishes toretain control over how the parking violation information flows back tothe parking operator. This can be accomplished by sending the vehicleviolation data back to the Pay-by-Plate system by using the API providedby server system 140. The Pay-by-Plate system will then have totalcontrol over how the list of “offending” vehicles is presented to theparking operator (such as by way of a custom software applicationdeveloped by the Pay-by-Plate system vendor).

A significant difference between this integration method and integrationwith the Pay-by-Plate system via the API provided by server system 140is that this integration method requires minimal development effort onthe part of the Pay-by-Plate system vendor. Since all the keyinformation to be presented to the parking operator is already suppliedby the API provided by server system 140 without any further analysisrequired, this information can simply be displayed or forwarded to theparking operator with minimal software development. This integrationapproach represents a “pull” followed by a “push” model.

There are two main reasons why use of a Live Operator team as part ofthe system illustrated in FIG. 1 warrants consideration and may offer apractical option to offer to prospective parking operators as a valueadded feature.

First, although server system 140 autonomously receives and processesimages obtained from the identification and destination cameras, thereare typically live operators who monitor obtained images and softwareexecuting on server system 140. These live operators can reviewoperation of server system 140 and take appropriate actions on anexception basis if required. For example, when unexpected problemsoccur, Live Operators can review images related to the problems anddeduce information that server system 140 may not be configured tohandle.

An advantage with the networked architecture supported by server system140 is that a single Live Operator team can oversee or augmentoperations at multiple sites regardless of their respective geographiclocations or the fact that the facility operations may be owned bydifferent parking operations or even competitors. As a result, the LiveOperator option can be a scalable and cost-effective approach forparking operators. Furthermore, by using IP multicast or similarstandard TCP/IP protocol, a video stream can effectively be sentsimultaneously to multiple destinations, thereby allowing the LiveOperator option to be added at a future date in addition to localon-site monitoring staffed by the parking operators.

Second, depending on the available technology, Pay-and-Display andcurbside meter configurations discussed above may present significantreal life challenges. Not only are cameras with a high level of zoom arerequired, there are also potential problems such as that an angle ofview from a particular video camera may not be sufficient to properlydetect an expiration flag of a traditional curbside parking meter, or abar code on a Pay-and-Display ticket on a vehicle dashboard may befacing away from any of the video cameras due to the physical threedimensional shape of the dashboard and the placement of the ticketitself.

With the Live Operator option, server system 140 can generate exceptionevents to the Live Operator when server system 140 fails toautomatically detect a key piece of information such as license platenumber, status of an expiration flag on a traditional parking meter, orbar code/printed information on a Pay-and-Display ticket. Such anexception event may trigger a Live Operator to take a number of specificactions, including but not limited to Rewind/FastForward/Pause/Freeze-Frame FWD & RWD specific footages captured byidentification and/or destination cameras, zoom in/out on particularareas of interest in the captured footages, Pan/Tilt/Zoom video camerasin real-time in order to obtain enhanced images, and dispatch on-sitepersonnel to perform inspection of a vehicle or parking meter, eithervisually of by use of mobile device 150 to obtain enhanced images foruse by server system 140.

In an embodiment, server system 140 is configured to correlate a mailingaddress of a registered owner of a vehicle with its license platenumber, allowing a parking operator to collect parking citation revenue.Red-light camera systems utilize such a mechanism to have citationtickets mailed to registered owners of the vehicles. In connection withsuch functionality, parking operators are in three different categories.

First, in the event that the parking operator is part of thecity/municipality who may already have access to DMV records, serversystem 140 only needs to supply a list of Vehicle License Plate Numberswith state/province information along with parking violation details(such as date, time, and location) in order for citations to be mailedto registered owners.

Second, for private parking operators with no pre-established access toDMV records, there are a number of legislated ways to obtain personalinformation, including name and address, tied to a Vehicle License PlateNumber. These methods vary from state to state (and provinces). As anexample, in the State of New York, private parking operators are allowedto gain access to such information by use of Driver's Privacy ProtectionAct (DPPA) form MV-15DPPA “for use in connection with the operation ofprivate toll transportation facilities”, “including companies thatoperate parking facilities for the purpose of providing notice to theowners of vehicles who have used the facility.” For some jurisdictions,name and address information may be released online, and server system140 may be configured to obtain and process such information. In somejurisdictions, name and address information of a third-party vehicleowner may not be released online, but instead must be obtained either bymail or in person. In such jurisdictions, private parking operators maywish to obtain name and address information on a batched basis (such asdaily, every 2 or 3 days, or weekly).

Third, for private parking operators who prefer real time access toregistered owner information, a number of private Vehicle License NumberPlate databases are available via the Internet. The main advantage ofthis approach is that name and address information can be immediatelyand automatically retrieved with no manual operation required.

As noted in connection with FIG. 1, end user systems 170 can includein-vehicle systems. In an embodiment, new vehicle dashboards straightfrom car manufacturers can include an indicator icon, similar to otherdashboard lights such as “check engine,” parking-brake warnings, orcruise-control indicator lights. This indicator icon is normally notilluminated when vehicle 130 is driving on the highway or not inside aperimeter serviced by server system 140. As vehicle 130 approachesparking lots or curbside parking areas, the indicator icon may light upin amber advising an end user of the availability of automated parkingservices provided by server system 140. When vehicle 130 pulls into adestination location, such as a parking space, the indicator icon turnsred to notify the end user that parking has not been paid. Once parkinghas been paid (whether by the afore-mentioned Pay-by-Phone, curbsidemeter, Pay-by-Space, or other techniques), the indicator icon turnsgreen to signify parking has been paid. In an embodiment, these visualindications may be accompanied by audio notifications, which may besimple chimes or voice announcements. Audio output may be performed byproviding audio information or data to a vehicle entertainment unit.

As this invention enables many aspects of parking payment andenforcement processes to be automated, it is important to providefeedback to end users as to what the server system 140 is doing or aparticular state server system 140 has associated with vehicle 130 (forexample, whether vehicle 130 has committed a parking violation). Endusers can then react accordingly to the feedback provided. Whileoperating within a highly automated payment and enforcement system, itis a reasonable expectation that end users or registered owners do notget penalized for minor or administrative errors. Feedback provided viaa simple interface such as the dashboard indicator icon can greatlyalleviate this concern.

For example, as vehicle 130 parks, if the indicator icon remains redafter some time, the end user realizes an exception event has takenplace. It may be that a credit card number on file has expired andtherefore server system 140 is not able to apply a parking charge. Thisfeedback provides the end user the opportunity to, for example, visit awebsite provided by or in connection with server system 140 to queryhis/her account status and correct the error.

There are a number of potential sources of information by which thedashboard indicator icon can get its information from, includingbuilt-in cellular data or WiFi vehicle connectivity, or smart sensors(discussed below). For older vehicles without factory-installedindicator icons, end users may be provided with a retrofit kit in theform of a visual module which receives information by way of Bluetooth,cellular data, or WiFi connectivity.

In an embodiment, the indicator icon feature receives GPS location datafrom third-party in-vehicle subsystems via a vehicle wiring harness. TheGPS data allows the indicator icon feature to determine whether vehicle130 is currently in a paid parking zone.

Similar to the vehicle dashboard icon, smart sensors can be designedinto new vehicles straight from the factory. Using technology similar tothe ZigBee IEEE 802.15.4 standard, low cost, low power, short rangewireless mesh networks can be created among vehicles as well as betweenvehicles and parking equipment. Each vehicle becomes a node, and a nodecan communicate with a distant end-point by relaying data throughintermediate nodes by forming a mesh network. Each smart sensor containsa unique identifier, such that license plate information is no longercritical. In an embodiment, smart sensors may be configured tocommunicate with server system 140 via a cellular data connection and/orprovide Pay-by-Phone functionality, whereby payment for use of adestination location may be made via the smart sensor.

Many applications are possible with the introduction of such smartsensors. For example, the smart sensors can include a GPS receiver orcommunicate with an in-vehicle GPS to determine a present location ofvehicle 130 and report the present location to server system 140. Withsome GPS techniques, such as assisted GPS, a location determined via GPSmay have enough accuracy and precision to determine the vehicle ismaking use of a particular destination location. These sensors can alsocommunicate with parking meters and in essence report the presence ofthe vehicle and how long it has been parked for. This parkinginformation can be relayed to a distant device/network through the meshnetwork so the parking equipment does not need to be located within thevicinity of the vehicle. For vehicles equipped with smart sensors, useof the identification and destination cameras is not necessary, as theembedded smart sensor already provide a unique identity of the vehicleto server system 140.

In an embodiment, vehicles may be equipped with a near-fieldcommunication (NFC) device, such as an RFID chip. For example, such acommunication device may be included in a mirror hanger. In anotherexample, an RFID chip may be embedded in a ticket provided by aPay-and-Display device. By obtaining a vehicle identification from anear-field communication device, use of images captured byidentification cameras, such as images including license plateinformation, may not be required by server system 140 and/or may be usedto verify information received from an NFC device corresponding to anobserved vehicle.

In an embodiment, the smart sensors can sense the status of othervehicles nearby (such as, but not limited to, whether they are moving,parked, and for what duration they have been parked) and report thestatus to server system 140. The end result is that server system 140 isaware as to where almost any smart sensor-equipped vehicle is parked andfor how long. By combining this information with parking paymentinformation from either third-party systems or turnkey platforms, it ispossible to determine the identity of vehicles that did not pay forparking or with expired parking, and have parking violation ticketsmailed to the registered owners or even charging a parking fine on thespot (for example, by charging a credit card on file).

In an embodiment, a destination camera can be provided with laser statusnotification. Similar to laser logo devices seen at shopping malls andretail outlets, the laser status notification feature shines anonharmful flashing red light enveloping a parked vehicle to notify anend user upon return that parking was not paid or has expired. This isakin to returning to a vehicle and seeing a parking ticket clipped underthe windshield wiper under the traditional manual parking enforcementmodel. As a destination camera is typically located at elevation withrespect to vehicles, an included laser status notification module has abird's eye view to below destination locations to facilitate shininglaser notification onto vehicles and destination locations.

In an embodiment, laser status notification feature replaces the need tomail out paper parking violation tickets. Once an end user receives alaser notification for a parking violation, the end user can then beexpected to pay the parking fine through a number of media including,but not limited to, an online website or Pay-by-Phone. Furthermore,laser status notifications can be captured by the destination camera toprovide later evidence of the parking violation and notification.

In an embodiment, dynamic real-time Individual Parking Space Reservationis a feature made possible by an online/phone/smart phone parking spacereservation system, and may be further enhanced with the laser statusnotification feature. While currently some vendors offer parking spacereservation feature, the feature is limited to space reservation in thegeneral vicinity of parking spaces (for example, somewhere on a parkinglot) as there is no practical method of physically reserving a single,particular parking space in an automated fashion. In addition, thereserved spaces are typically held open for a long period of timebecause there is also no practical way of knowing when a reservedparking spot becomes available again after the driver who reserved thespace has parked and subsequently driven off.

In contrast, the dynamic real-time Individual Parking Space Reservationfeature allows end users to pinpoint the exact parking space they desire(for example, a space closest to a shopping mall during Christmasshopping period), for a specified period of time (for example, adestination location may be reserved for use during some day in thefuture). Once a parking space has been reserved, a laser notificationmay be shined on the parking spot to indicate that the destinationlocation is reserved (providing, for example, a red color as a warningto other drivers). The use of a destination location during a reservedperiod by another vehicle will result in a parking violation. Thisenables a parking operator to increase revenue by offering bothtiered/premium parking spots, and advance reservation of parking spaces.

In an embodiment, a parking operator may designate particulardestination locations as restricted only for use by vehicles for which areservation has been made. In this embodiment, server system 140 maydetermine a number of available reserved destination locations for agiven time, and price reservations according to how many unreserveddestination locations remain available. In an embodiment, server system140 may be configured to allow buyers to submit competing bids for theuse of reserved destination locations.

In an embodiment, a Pay-by-Phone feature can be offered either directlyon a turnkey platform or through a partnership with a third-partyvendor. The end result is that end users can pay for parking by callinga particular IVR phone line, input a destination location number orvehicle license plate number, and a duration of parking. In anotherembodiment, Pay-Online is a variation along the same theme to supportonline payment for parking, typically via a website accessible via aweb-enabled smartphone.

In an embodiment, a “push” technology feature can be incorporated whereas soon as an end user parks vehicle 130, the end user receives a textmessage or other notification prompting for an affirmative reply tocharge a specific parking rate until vehicle 130 leaves a destinationlocation. Server system 140 identifies vehicle 130 via an identificationcamera, and further determines that vehicle 130 is associated with apre-registered end user with credit card or other billing informationand a cellular phone number or other contact information on record. Onceserver system 140 determines via a destination camera that vehicle 130has parked in a destination location, server system 140 initiates a“push” sequence to actively notify and engage the driver for payment orconfirmation of payment. By simplifying the payment process, improvedparking revenue is realized.

In many traditional parking meter systems, if an end user pays for moretime than actually used by the end user, excess time remains on themeter for use by a following end user. In an embodiment of the disclosedsubject matter, as destination camera 125 allows server system 140 toaccurately determine when a vehicle leaves a destination location, any“leftover” paid parking time can be reset to zero. One exception istraditional curbside meters, as these typically do not have anycommunication capability or the intelligence to activate the expirationflag prior to time expiration.

Server system 140 may be configured to provide a push technologysoftware component that enhances the user experience when it comes topaying for parking. Push technology supports functions such as: (1)initiating a payment sequence via text message; (2) allowing apre-registered user to take a photo of their license plate, then providethe photo to server system 140 by way of, for example, email ortext-message; (3) allowing a non-registered user to take a photo oftheir license plate, then provide the photo to server system 140 by wayof, for example, email or text-message; and (4) allowing drivers to signup once to not get parking fees under a subscription-based model. Thepush technology component interacts with vehicle identifier detectionlogic, vehicle detection logic, database 141, and a payment gateway. Inconnection with item (1), based on a vehicle identifier of the vehiclethat has just begun use of a destination location such as a parkingspace, the push technology component pushes a text message indicatingthat use of the destination location has begun, to a mobile phone numberregistered with server system 140. When the driver assents to the textmessage by replying with “yes”, a credit card associated with theregistered account may then get charged an appropriate usage fee, suchas, but not limited to, a flat-rate parking fee for use of thedestination location. In connection with item (2), a driver who haspre-registered with server system 140 may take photos of their licenseplate, then email or text-message the photo to server system 140. Acredit card associated with a registered account for the pre-registereduser then gets charged a usage fee. In connection with item (3), forvehicles bearing license plates not registered with server system 140, adriver may take a photo of the license plate, then email/text-messagethe photo to server system 140. Where a mobile phone service providersupports and provides access to such a billing feature, usage chargesmay be added to a user's monthly mobile device bill. In connection withitem (4), a driver may sign up once to automatically pay for all usagefees (or at least all parking fees) with a registered credit card. Thepush technology software component may also be configured toautomatically send a text message to a user In the event that serversystem 140 determines inappropriate use of a destination location, suchas parking a vehicle in a “no parking” area.

Server system 140 may be configured to provide a parking violationpre-processing logic software component that examines all occurrences ofdestination location usages newly identified as violations and assignsthem to a corresponding destination as specified in a systemconfiguration, such as the license plate lookup module or the licenseplate forward module, discussed below. The parking violationpre-processing logic software component interacts with database 141, thesystem configuration utility, and third-party APIs. The parkingviolation pre-processing logic software component may be configured to:(1) enqueue violations identified by other software processes includedin server system 140; (2) examine the next violation in the queue; (3)validate the destination location usage is a violation by verifyingrules established via the system configuration utility in connectionwith the destination location and detected use of the destinationlocation by a vehicle; (4) assign the violation to license plateprocessing logic; and (5) examine any other enqueued violations or awaitan identification of a later violation.

Server system 140 may be configured to determine whether a vehicle hasan expired tag from an identification image. The identification imagemay be captured using a fixed Identification camera 120, although attypical distances and vehicle speeds, and the small size of the typicalexpiration date indicator on a license plate, determining whether avehicle bears an expired tag from the identification image alone isgenerally not possible. With a PTZ-capable camera zoomed into a licenseplate of a stationary vehicle, either with an Identification camera ordestination camera, it may be possible to zoom into a license plate toobtain enough detail to determine whether a vehicle bears an expired tagfrom the obtained image alone. In an embodiment, the identificationimage instead can be captured via a mobile or handheld camera,including, for example, a camera included in a smartphone or mobiledevice 150. An identification image obtained from such sources often hasenough detail to determine whether a vehicle bears an expired tag fromthe identification image alone. In an embodiment, rather than use animage alone to determine if a tag is expired, server system 140 may beconfigured to query, via an API to a municipal database, determinewhether a vehicle registration for a particular license plate number isexpired. Server system 140 may be configured to, in the event itdetermines a vehicle is bearing an expired tag, report the vehicle to amunicipality.

Server system 140 may be configured to provide a license plateprocessing logic software component that takes an identified destinationlocation usage violation, such as a violation assigned by the parkingviolation pre-processing logic component, and converts it into anactionable item for either a Registered Owner (RO)/user of a vehicleinvolved in the destination location usage violation, or for a parkingoperator responsible for the destination location. For example, theactionable item can be in the form of a parking citation automaticallymailed to the registered owner, or a license plate number displayed on ascreen at a parking operations office. The license plate processinglogic software component interacts with the parking violationpre-processing logic software component, system configuration utility,and database 141. The license plate processing logic component may beconfigured to check one or more business rules specified via the systemconfiguration utility in connection with the destination location todetermine the appropriate actionable item. For example, if a businessrule specifies license plate lookup, a specific type of license platelookup may also be specified, such as via an API provided by amunicipality, or an API provided by a private business. Then licenseplate lookup is performed as specified. Also, business rules mayidentify sub-options such as mailing out parking tickets to a registeredowner via API, or just providing a mailing address for a registeredowner and any other associated information for the violation to theparking operator. In another example, if a business rule specifieslicense plate information forwarding, the license plate processing logiccomponent checks to see the specific type of license plate informationforwarding specified, and forwards the license plate informationaccordingly.

In order to utilize a license plate lookup API option for the purpose ofidentifying a mailing address for a registered owner from a licenseplate vehicle identifier, a license plate database, such as one providedby a private business, must exist and be available for searches forlicense plates belonging to a particular state/province for the licenseplate. Also, use of such a database must be reasonable in terms of anyfees involved to such lookup is commercially viable. In the event thateither condition above does not materialize, an alternative option mayinclude providing the license plate in question to a municipal body,which typically has access to license plate records via the local policebranch or otherwise.

The license plate processing logic software component may include alicense plate lookup and followup submodule. The license plate lookupand followup submodule may be configured to make an API call to alicense plate database to retrieve the mailing address for a registeredowner of a vehicle. Examples of such databases include an HTTP-basedDial-in Search Account API provided by the New York State Department ofMotor Vehicles, database services provided by the American Associationof Motor Vehicle Administrators (AAMVA) or other insurance companies,and online collections of public records, such as PublicData.com(although many such collections are less reliable than government orcommercially managed databases, thus possibly requiring additionalefforts to ensure information is valid and accurate).

Also, the license plate lookup and followup submodule may be configuredto automatically mail a parking citation to a registered ownerassociated with a vehicle that has misused a destination location. Forexample, the process of mailing may be automated via an API call to aprinting/mailing service such as, but not limited to, L-Mail.com.

In an embodiment, the license plate lookup and followup submodule may beconfigured to create, maintain, and review records in database 141 tosee if a given vehicle identifier (such as a license plate) isassociated with prior misuse of a destination location within aspecified period of time (which is configurable via, for example, thesystem configuration utility), If so, the license plate lookup andfollowup submodule may presume that the previously obtained addressremains valid and accurate, skip the API call, and mail a citation usingthe previously obtained mailing address in the system in order to loweroperating expenses associated with API calls. In the event that theparking citation has not been paid after a specified number of days(actual duration also configurable via, for example, the systemconfiguration utility) or the citation is returned as undeliverable, asubsequent API call may be made to obtain a current address.

In an embodiment, a parking violation, such as, but not limited to, whenpaid parking expires or a vehicle is parked without payment, serversystem 140 may be configured to notify one or more enforcement officerswith a location of a destination location, such as a longitude andlatitude or an approximate street address. An enforcement officer thenmay go to the offending vehicle and issue a citation on the spot. In anembodiment, server system 140 may be configured to identify and/orlocate an individual enforcement officer who is best suited to deal witha particular offending vehicle. Factors for making this identificationmay include, for example, the distance from the offending vehicle orworkload of an enforcement officer, or whether an offending vehicle isto be towed (for example, where an offending vehicle must be promptlyremoved). In an embodiment, offending vehicles may have to be enqueuedwhere, for example, there are too few enforcement officers or aparticular enforcement officer has multiple offending vehicles tohandle. In an embodiment, multiple enforcement officers may be notifiedof nearby offending vehicles, and individual enforcement officers mayaccept to handle particular offending vehicles. Server system 140 mayalso be configured to track enforcement officer performance, which maybe used in connection with a reward or quota system for the enforcementofficers. In an embodiment, an enforcement officer may be equipped witha mobile device 150, and mobile device 150 may be configured to preprinta citation for an offending vehicle to reduce an amount of time taken todeliver the citation to the vehicle.

Server system 140 may be configured to provide dynamic variable-rateparking fees. In an embodiment, server system 140 may record differentparking rates for a destination location which vary according to time ofday and/or day of the week, for example. In the event that a vehicleoccupies a destination location over a period of time in which first andsecond rates apply, server system 140 may be configured to determine atotal fee for use of the destination location based the first rate beingapplied from the beginning of the use of the destination location untilthe second rate applies, and the second rate being applied for theremaining time the vehicle occupies the destination location. In anembodiment, server system 140 may be configured to determine a number ofvehicles which may be accommodated by the destination locationsavailable in a given area, and base a fee or rate for parking on thedetermined number of vehicles. For example, as server system 140determines this number decreases (for example, because there are fewerremaining destination locations), it may apply a higher rate forvehicles which begin to make use of the few remaining locations.

Server system 140 may be configured to allow a parking provider, via anAPI or the configuration utility, specify particular events for which aspecified parking rate or flat fee is to be applied for specifieddestination locations. For example, if a concert occurs from 7-11 PM ata venue associated with or near a parking facility, the parking facilitymay levy a flat $10 fee for vehicles parked after 4 PM that sameevening.

Server system 140 may be configured to provide a no-fine feature, wherea driver may leave a vehicle parked in a destination location for aslong as they wish, and have a credit card or other account billed foraccording to rules established for the destination location (such as,but not limited to, per-hour or flat rate per-day fees). In anembodiment, a text message or email may be pushed or otherwise sent tothe driver indicating that the no-fine feature is available for thedestination location, and allowing the driver to elect use of theno-fine feature, such as by replying with a message such as “yes” or “nofine”. In an embodiment, server system 140 may be configured togenerally make the no-fine feature available for all unrestricteddestination locations in specified areas or throughout the destinationlocations tracked by server system 140. However, server system 140 maybe configured not to offer the no-fine feature for a restricteddestination location, such as, for example, a destination locationsubject to a 3-6 PM “no parking” rush hour restriction.

In an embodiment, server system 140 may be able to perform profilingbased on 3D outline of the vehicle. For example, such profiling may beable to roughly profile a vehicle as belong to a category such as, butnot limited to, sport utility vehicle (SUV), van, sedan, coupe, ormotorcycle. In addition, a vehicle may be categorized as having a lightor dark color. The vehicle profiling can serve non-parking applications,such as, but not limited to, assisting police tracking of stolenvehicles within a certain vicinity, Amber alert, Homeland Securityneeds. In an embodiment, the vehicle profiling can be used to provideadditional vehicle characteristics for comparisons performed in trackinga vehicle from one tracking camera to another. In an embodiment, thevehicle profiling may provide an additional layer of confirmation of anidentity of a vehicle for an automated parking citation mailing process.In an embodiment, manual 3D vehicle profiles are provided to serversystem 140. For example, one manual 3D vehicle profile for a ChevroletImpala may describe a ratio of 1:2:1 for height of hood to height ofpassenger compartment to height of trunk.

In an embodiment, a laser transmitter, which may be integrated with anactuated beam-deflecting mirror, may be configured to shine a laser beamonto one or more destination locations. In an embodiment, the laser beammay shine on a retroreflector mounted on or around a destinationlocation to provide a return signal to a receiver associated with thelaser transceiver. When a vehicle occupies a destination locationilluminated by the laser beam, the presence of the vehicle is detectedby interruption of the laser beam or a return signal. In response tothis detection, server system 140 may be configured to obtain adestination image and/or an identifier image with a nearby camera.

In an embodiment, server system 140 may be configured to, in addition tothe aforementioned uses involving vehicle identification, also use anidentification camera as a “red light camera” for the detection ofevents in which a vehicle incorrectly proceeds through an intersectioncontrolled by a traffic light. For example, as a traffic light changesfrom amber to red, vehicles incorrectly proceeding through a late amberor red light may be identified, along with a captured vehicle licenseplate, such that server system 140 is configured to simultaneouslyutilize an imaging camera for both purposes. Alternatively or inaddition, server system 140 may be configured to use an identificationcamera to detect “block the box” moving violations, where a vehicleremains within an intersection during a red light for the vehicle'sdirection of travel, which may lead to gridlock high-traffic situations.

In an embodiment, server system 140 may be configured to providereal-time vehicle tracking services for third parties such as, but notlimited to, repossession companies (for locating vehicles withoutstanding loan payments), law enforcement, homeland security. Serversystem 140 may provide an API by which such third-party entities mayregister vehicles of interest by, for example, a vehicle identifier suchas a license plate. In the event that server system 140 determines thepresence of a vehicle with the vehicle identifier, the registering partymay be notified. Notifications may be provided in connection with aninitial detection of the vehicle and/or when the vehicle is determinedto have stopped. In an embodiment, records of the movements of aregistered vehicle may be retained by server system 140, and madeaccessible, for example, via a map-based web interface, which may beconfigured to present updated information in real-time. The API may alsoallow a third party to designate an area of interest, for which serversystem 140 is configured to perform identification and tracking of anyvehicles determined to have entered the area of interest. Server system140 may also be configured to provide images of registered vehiclesobtained by the tracking cameras. In an embodiment, any archivedtracking information, such as the past use of destination locations, mayalso be made available to reconstruct a person's whereabouts over time(assuming the person is connected with the vehicle). In an embodiment,server system 140 may be configured to obtain vehicle images includingan image of a driver's face, and may further be configured to performfacial recognition techniques to confirm facial characteristics of thedriver correspond to a person of interest.

In an embodiment, server system 140 may be configured to determine andprovide traffic count information, such as the number of vehiclesdetermined to have traversed a given portion of roadway. Suchinformation may be employed for purposes including, but not limited to,dynamic control of traffic systems such as traffic lights in response tocurrently observed traffic counts, and for determining constructionrequirements based on traffic counts observed over time. The trafficcount information may include and/or be filtered on the basis of timeand/or date, and/or determined vehicle characteristics, including, butnot limited to, direction of travel and vehicle size or type (such asSUV, truck, car, or motorcycle). Additionally, server system 140 maydetermine vehicle counts in connection with a direction of entry and/orexit for a portion of roadway, such as, for example, how many vehicleswere northbound or southbound upon exiting an eastbound portion ofroadway, based on a direction each vehicle exits an intersection. In anembodiment, server system 140 may be configured for use in controllingaccess to and/or levy a toll upon vehicles by type, such as assessing afee for the use of specified roadways by large trucks.

FIG. 5 is a block diagram that illustrates a computer system 500 uponwhich aspects of the invention may be implemented. Computer system 500includes a bus 502 or other communication mechanism for communicatinginformation, and a processor 504 coupled with bus 502 for processinginformation. Computer system 500 also includes a main memory 506, suchas a random access memory (RAM) or other dynamic storage device, coupledto bus 502 for storing information and instructions to be executed byprocessor 504. Main memory 506 also may be used for storing temporaryvariables or other intermediate information during execution ofinstructions to be executed by processor 504. Computer system 500further includes a read only memory (ROM) 508 or other static storagedevice coupled to bus 502 for storing static information andinstructions for processor 504. A storage device 510, such as a magneticdisk or optical disk, is provided and coupled to bus 502 for storinginformation and instructions.

Computer system 500 may be coupled via bus 502 to a display 512, such asa cathode ray tube (CRT) or liquid crystal display (LCD), for displayinginformation to a computer user. An input device 514, includingalphanumeric and other keys, is coupled to bus 502 for communicatinginformation and command selections to processor 504. Another type ofuser input device is cursor control 516, such as a mouse, a trackball,or cursor direction keys for communicating direction information andcommand selections to processor 504 and for controlling cursor movementon display 512. This input device typically has two degrees of freedomin two axes, a first axis (e.g., x) and a second axis (e.g., y), thatallows the device to specify positions in a plane. Another type of userinput device is a touchscreen, which generally combines display 512 withhardware that registers touches upon display 512.

The invention is related to the use of one or more computer systems,such as computer system 500, collectively configured and/or programmedto perform implementing the techniques described herein. According toone embodiment of the invention, those techniques are performed bycomputer system 500 in response to processor 504 executing one or moresequences of one or more instructions contained in main memory 506. Suchinstructions may be read into main memory 506 from anothermachine-readable medium, such as storage device 510. Execution of thesequences of instructions contained in main memory 506 causes processor504 to perform the process steps described herein. In alternativeembodiments, hard-wired circuitry may be used in place of or incombination with software instructions to implement the invention. Thus,embodiments of the invention are not limited to any specific combinationof hardware circuitry and software.

The term “machine-readable medium” as used herein refers to any mediumthat participates in providing data that causes a machine to operationin a specific fashion. In an embodiment implemented using computersystem 500, various machine-readable media are involved, for example, inproviding instructions to processor 504 for execution. Such a medium maytake many forms, including but not limited to, non-volatile media,volatile media, and transmission media. Non-volatile media includes, forexample, optical or magnetic disks, such as storage device 510. Volatilemedia includes dynamic memory, such as main memory 506. Transmissionmedia includes coaxial cables, copper wire and fiber optics, includingthe wires that comprise bus 502. Transmission media can also take theform of acoustic or light waves, such as those generated duringradio-wave and infra-red data communications. All such media must betangible and/or nontransitory to enable the instructions carried by themedia to be detected by a physical mechanism that reads the instructionsinto a machine.

Common forms of machine-readable media include, for example, a floppydisk, a flexible disk, hard disk, magnetic tape, or any other magneticmedium, a CD-ROM, any other optical medium, punchcards, papertape, anyother physical medium with patterns of holes, a RAM, a PROM, and EPROM,a FLASH-EPROM, any other memory chip or cartridge, a carrier wave asdescribed hereinafter, or any other medium from which a computer canread.

Various forms of machine-readable media may be involved in carrying oneor more sequences of one or more instructions to processor 504 forexecution. For example, the instructions may initially be carried on amagnetic disk of a remote computer. The remote computer can load theinstructions into its dynamic memory and send the instructions over atelephone line using a modem. A modem local to computer system 500 canreceive the data on the telephone line and use an infra-red transmitterto convert the data to an infra-red signal. An infra-red detector canreceive the data carried in the infra-red signal and appropriatecircuitry can place the data on bus 502. Bus 502 carries the data tomain memory 506, from which processor 504 retrieves and executes theinstructions. The instructions received by main memory 506 mayoptionally be stored on storage device 510 either before or afterexecution by processor 504.

Computer system 500 also includes a communication interface 518 coupledto bus 502. Communication interface 518 provides a two-way datacommunication coupling to a network link 520 that is connected to alocal network 522. For example, communication interface 518 may be anintegrated services digital network (ISDN) card or a modem to provide adata communication connection to a corresponding type of telephone line.As another example, communication interface 518 may be a local areanetwork (LAN) card to provide a data communication connection to acompatible LAN. Wireless links may also be implemented. In any suchimplementation, communication interface 518 sends and receiveselectrical, electromagnetic or optical signals that carry digital datastreams representing various types of information.

Network link 520 typically provides data communication through one ormore networks to other data devices. For example, network link 520 mayprovide a connection through local network 522 to a host computer 524 orto data equipment operated by an Internet Service Provider (ISP) 526.ISP 526 in turn provides data communication services through the worldwide packet data communication network now commonly referred to as the“Internet” 528. Local network 522 and Internet 528 both use electrical,electromagnetic or optical signals that carry digital data streams. Thesignals through the various networks and the signals on network link 520and through communication interface 218, which carry the digital data toand from computer system 500, are exemplary forms of carrier wavestransporting the information.

Computer system 500 can send messages and receive data, includingprogram code, through the network(s), network link 520 and communicationinterface 518. In the Internet example, a server 530 might transmit arequested code for an application program through Internet 528, ISP 526,local network 522 and communication interface 518.

The received code may be executed by processor 204 as it is received,and/or stored in storage device 510, or other non-volatile storage forlater execution. In this manner, computer system 500 may obtainapplication code in the form of a carrier wave.

FIG. 7 illustrates the unmanned aerial vehicle 700 that includes atleast one camera 702. The unmanned aerial vehicle 700 can includeautonomous aircraft and/or remotely piloted aircraft includingairplanes, helicopters, miniature UAV or small UAVs (SUAV), micro airvehicles (MAV), and the like. The unmanned aerial vehicle 700 mayinclude at least one camera that can be used in place of or inconjunction with one or both of the identification camera 120 anddestination camera 125. Images captured by the unmanned aerial vehicle700 may be forwarded to server system 140 and subsequently processed bythe server system 140.

The camera 702 on the unmanned aerial vehicle 700 may be used to captureimages of a vehicle 704. The unmanned aerial vehicle 700 may first belocated at a first location 706 where the camera 702 has a first fieldof view 708. The first field of view 708 may show one perspective of thevehicle 704 in view an identifier of the vehicle 704 may not beidentifiable. In response, the unmanned aerial vehicle 700 may berepositioned from the first location 706 to a second location 710 havinga second field of view 712, which may show a different view of thevehicle 704. The identifier of the vehicle 704 may be identifiable basedon the images received when the unmanned aerial vehicle 700 is in thesecond location 710.

FIG. 8 illustrates a process 800 for tracking a vehicle 704 using anunmanned aerial vehicle 700. At step 802, the server system 140 mayreceive one or more first images captured by a camera 702 of theunmanned aerial vehicle 700. The at least one first images may becaptured by the camera 702 while the unmanned aerial vehicle 700 is at afirst location 706 where the unmanned aerial vehicle 700 may have afirst field of view 708 showing a vehicle 704. The unmanned aerialvehicle 700 may forward the at least one first images to the serversystem 140.

At step 804, the unmanned aerial vehicle 700 and/or server system 140may determine a first characteristic of the vehicle based on the one ormore first images captured by the camera 702. The first characteristicmay be at least one static characteristic and/or at least one dynamiccharacteristic of the vehicle 704. A static characteristic may be acharacteristic of the vehicle 704 that does not change over time and mayinclude one or more of a vehicle make, a vehicle model, a vehicle year,a vehicle color, a vehicle outline, a vehicle size, a vehicle dimension,and other static information related to the vehicle 704. A dynamiccharacteristic may be a characteristic of the vehicle 704 that changesover time and may include one or more of a lane identifier, a direction,a ground speed, an acceleration, and other dynamic information relatedto the vehicle 704. The static and dynamic characteristics may bedetermined using methods described above with respect to the imagescaptured by one or both of the identification camera 120 and destinationcamera 125.

At step 806, the unmanned aerial vehicle 700 and/or server system 140may determine that a unique identifier of the vehicle 704 is notidentifiable in at least one of the one or more first images. A uniqueidentifier can include a license plate number, a one-dimensional barcode, a matrix bar code, a parking permit identification number, adisabled placard identification number, a vehicle registration tag, oranother identifier associated with the vehicle 704. In some aspects, theunique identifier of the vehicle 704 may not be visible in at least oneof the one or more first images, while in other aspects, the uniqueidentifier of the vehicle 704 may not be readable in at least one of theone or more first images due to the at least one of the one or morefirst images being out of focus, for example. The methods describedabove with respect to optical character recognition (OCR) may beimplemented to determine whether an unique identifier of the vehicle 704is identifiable.

At step 808, the server system 140 may cause the unmanned aerial vehicle700 to reposition from the first location 706 to a second location 710that is different from the first location 706. The unmanned aerialvehicle 700 may receive instructions to reposition to a second location710 where the camera 702 may be able to better capture the uniqueidentifier of the vehicle 704. For example, the first field of view 708may show a top of the vehicle 704 where a license plate is not visible,while the second field of view 712 may show a front region or a rearregion of the vehicle 704 where the license plate is visible. In someaspects, the unmanned aerial vehicle 700 may reposition from the firstlocation 706 to the second location 710 without communication with theserver system 140. The distance between the unmanned aerial vehicle 700and the vehicle 704 may be the same when the unmanned aerial vehicle 700is at the first location and when the unmanned aerial vehicle 700 is atthe second location.

In some aspects, to reposition the unmanned aerial vehicle 700, theunmanned aerial vehicle 700 and/or server system 140 may determine afirst path to the second location 710 from the first location 706. Thefirst path may be a straight-line path or the shortest distance pathbetween the first location 706 and the second location 710. The unmannedaerial vehicle 700 and/or server system 140 may determine that anobstruction is present along the first path. This determination may bedone by using visual information captured by a camera 702 located on theunmanned aerial vehicle 700 or known obstruction information located ina database of the server system 140. Obstructions may include, forexample, aerial vehicles, telephone or service lines, bridges,overpasses, trees, signs or billboards, and any other objects that mayimpede a flight path of the unmanned aerial vehicle 700. In response tothe determination that an obstruction is present along the first path,the unmanned aerial vehicle 700 and/or server system 140 may determine asecond path from the first location to the second location 710 that doesnot overlap and/or include the obstruction. The unmanned aerial vehicle700 may then reposition to the second location 710 by following thesecond path.

In some aspects, the vehicle 704 may be moving while being tracked bythe unmanned aerial vehicle 700. To be able to image the uniqueidentifier at a future point in time, the unmanned aerial vehicle 700and/or server system 140 may determine or predict where the vehicle 704will be at the future point in time by using at least one dynamiccharacteristic of the vehicle 704, such as speed, acceleration,direction, or another dynamic characteristic. Based on location and/ordynamic characteristic information of both the unmanned aerial vehicle700 and the vehicle 704, the unmanned aerial vehicle 700 and/or serversystem 140 may be able to determine a second location where the uniqueidentifier will be visible in images captured by the camera 702 havingthe second field of view 712. In some aspects, the unmanned aerialvehicle 700 and/or server system 140 may also determine or predict thefuture location of the unique identifier of the vehicle 704specifically.

In some aspects, the unmanned aerial vehicle 700 may only reposition toimage the unique identifier of the vehicle 704 once certain criteria aremet, such as determining the vehicle 704 is exceeding a speed limit ordetermining that the vehicle 704 is a vehicle of interest. For example,the unmanned aerial vehicle 700 and/or server system 140 may determineat least one static characteristic and/or dynamic characteristic of thevehicle 704. The unmanned aerial vehicle 700 and/or server system 140may determine that a magnitude of the static and/or dynamiccharacteristic exceeds a first threshold or matches predeterminedcriteria. Specifically, the unmanned aerial vehicle 700 and/or serversystem 140 may determine a speed of the vehicle 704 exceeds a speedlimit or the color and/or shape of the vehicle 704 matches a vehicle ofinterest. The unmanned aerial vehicle 700 and/or server system 140 maythen cause repositioning of the unmanned aerial vehicle 700 from thefirst location 706 to the second location 710 to identify the uniqueidentifier of the vehicle 704.

In another aspect, the unmanned aerial vehicle 700 may only repositionto image the unique identifier of the vehicle 704 if a parking violationhas been committed. Examples of parking violations include failing tohave a parking permit, failure to display or have a visible parkingpermit, having an expired or invalid parking permit, having an expiredparking meter, parking in an unauthorized space, such as in a restrictedspace, a reserved space, or a no parking zone, parking on the sidewalk,parking in a fire lane, blocking another vehicle, or another parkingviolation. The unmanned aerial vehicle 700 and/or server system 140 maydetermine, based on at least one of one or more first images, that thefirst vehicle has committed a parking violation. The unmanned aerialvehicle 700 and/or server system 140 may then cause repositioning of theunmanned aerial vehicle 700 from the first location 706 to the secondlocation 710 to identify the unique identifier of the vehicle 704.

At step 810, the server system 140 may receive one or more second imagesfrom the unmanned aerial vehicle 700. Similar to the first images, theone or more second images may be captured by the camera 702 of theunmanned aerial vehicle 700. The images may be captured by the camera702 while the unmanned aerial vehicle 700 is at the second location 710imaging a different portion of the vehicle 704. The unmanned aerialvehicle 700 may forward the one or more first images to the serversystem 140.

At step 812, the unmanned aerial vehicle 700 and/or server system 140may determine a second characteristic of the vehicle based on the one ormore second images captured by the camera 702. The second characteristicmay be one or more of a static characteristic or a dynamiccharacteristic and include any of the characteristics mentioned abovewith respect to the first characteristic. The type of the secondcharacteristic may match the type of the first characteristic. Forexample, if the first characteristic was a speed of the vehicle 704 at afirst point in time, the second characteristic may be a speed of thevehicle 704 at a second point in time. In some aspects, the type of thesecond characteristic may be different from the type of the firstcharacteristic.

At step 814, the unmanned aerial vehicle 700 and/or server system 140may compare the second characteristic to the first characteristic. Insome aspects, the first characteristic may include a first staticcharacteristic and a first dynamic characteristic, and the secondcharacteristic may include a second static characteristic and a seconddynamic characteristic. The unmanned aerial vehicle 700 and/or serversystem 140 may compare the first static characteristic to the secondstatic characteristic and may compare the first dynamic characteristicto the second dynamic characteristic.

At step 816, the unmanned aerial vehicle 700 and/or server system 140may determine, in response to the comparison, that the secondcharacteristic is approximately equal to the first characteristic. Theunmanned aerial vehicle 700 and/or server system 140 may determine thefirst and second characteristics are approximately equal when adifference between magnitudes of the first and second characteristics ora ratio between the first and second characteristics is below apredetermined threshold.

For example, if the first characteristic is a vehicle speed determinedto be 60 mph and the second characteristic is a vehicle speed determinedto be 58 mph, the unmanned aerial vehicle 700 and/or server system 140may determine the two characteristics are approximately equal if thethreshold is +/−5 mph. In another aspect, the first characteristic andsecond characteristic may each include more than one static and/ordynamic characteristic. The unmanned aerial vehicle 700 and/or serversystem 140 may determine an approximate equivalence between each of thestatic and dynamic characteristics. At step 818, the unmanned aerialvehicle 700 and/or server system 140 may determine, in response to theapproximate equivalence between the first characteristic and the secondcharacteristic, that the vehicle 704 is present in the one or moresecond images.

At step 820, the unmanned aerial vehicle 700 and/or server system 140may determine and identify, based on at least one of the one or moresecond images, the unique identifier of the vehicle 704. The uniqueidentifier of the first vehicle may be a license plate number, aone-dimensional bar code, a matrix bar code, a parking permitidentification number, a disabled placard identification number, or avehicle registration tag associated with the first vehicle. The methodsdescribed above with respect to optical character recognition (OCR) maybe implemented by the unmanned aerial vehicle 700 and/or server system140 to determine the unique identifier. Other methods known in the artmay also be used to determine the unique identifier. In one aspect, theunique identifier may be stored in a non-volatile memory and/or databaseof the unmanned aerial vehicle 700 and/or server system 140. In anotheraspect, the unique identifier may be used in a method implemented by themobile device 150, onsite payment system 160, end user system 170,manual review system 180, and/or parking operation system 190 asdescribed above in reference with FIGS. 1-6.

In another aspect, the unmanned aerial vehicle 700 may handoff trackingof the vehicle 704 to a second camera. The second camera may be a cameraon another unmanned aerial vehicle, an identification camera 120, adestination camera 125, or another camera. The server system 140 mayreceive one or more third images from the second camera having a thirdfield of view and showing the vehicle 704. The server system 140 maydetermine an overlap area that defines the overlap between the one ormore first images received previously and the one or more third images.The server system 140 may determine a handoff zone within the overlaparea where the tracking of the vehicle 704 is passed from the secondcamera to the camera 702 on the unmanned aerial vehicle 700 or passedfrom the camera 702 on the unmanned aerial vehicle 700 to the secondcamera after the vehicle 704 enters the handoff zone.

In the foregoing specification, embodiments of the invention have beendescribed with reference to numerous specific details that may vary fromimplementation to implementation. Thus, the sole and exclusive indicatorof what is the invention, and is intended by the applicants to be theinvention, is the set of claims that issue from this application, in thespecific form in which such claims issue, including any subsequentcorrection. Any definitions expressly set forth herein for termscontained in such claims shall govern the meaning of such terms as usedin the claims. Hence, no limitation, element, property, feature,advantage or attribute that is not expressly recited in a claim shouldlimit the scope of such claim in any way. The specification and drawingsare, accordingly, to be regarded in an illustrative rather than arestrictive sense.

What is claimed is:
 1. A method of tracking a vehicle using an unmannedaerial vehicle, the method comprising: receiving, at a first time, oneor more first images from a first camera of a first unmanned aerialvehicle located at a first location, the one or more first images havinga first field of view and showing a first vehicle; determining, based onat least one of the one or more first images, a first characteristic ofthe first vehicle; determining that a unique identifier of the firstvehicle is not identifiable in at least one of the one or more firstimages; causing repositioning, in response to the determination that theunique identifier of the first vehicle is not identifiable in the atleast one of the one or more first images, of the first unmanned aerialvehicle from the first location to a second location, the secondlocation being different from the first location; receiving, at a secondtime that is after the first time, one or more second images from thefirst camera of the first unmanned aerial vehicle, the one or moresecond images having a second field of view and showing the firstvehicle, the second field of view being different from the first fieldof view; determining, based on at least one of the one or more secondimages, a second characteristic of the first vehicle; comparing thesecond characteristic of the first vehicle to the first characteristicof the first vehicle; determining, in response to the comparison of thesecond characteristic of the first vehicle to the first characteristicof the first vehicle, that the second characteristic of the firstvehicle is approximately equal to the first characteristic of the firstvehicle; determining, in response to the approximate equivalence of thesecond characteristic of the first vehicle and the first characteristicof the first vehicle, that the first vehicle is present in the one ormore second images; determining, based on at least one of the one ormore second images, the unique identifier of the first vehicle; andidentifying the unique identifier, wherein the steps of receiving,determining, causing repositioning, comparing and identifying areperformed by one or more processors.
 2. The method of claim 1, whereincausing repositioning of the first unmanned aerial vehicle from thefirst location to the second location comprises: determining a dynamiccharacteristic of the first vehicle; and determining, based on thedynamic characteristic of the first vehicle, the second location of thefirst unmanned aerial vehicle.
 3. The method of claim 2, wherein thedynamic characteristic of the first vehicle is at least one of a laneidentifier, a direction, a ground speed, and an acceleration of thefirst vehicle.
 4. The method of claim 2, wherein determining the secondlocation of the first unmanned aerial vehicle comprises estimating alocation of the first vehicle at the second time based on the dynamiccharacteristic of the first vehicle.
 5. The method of claim 2, whereindetermining the second location of the first unmanned aerial vehiclecomprises determining a location of the unique identifier of the firstvehicle at the second time based on the dynamic characteristic of thefirst vehicle.
 6. The method of claim 1, wherein causing repositioningof the first unmanned aerial vehicle from the first location to thesecond location comprises: determining the second location of the firstunmanned aerial vehicle; and forwarding instructions to the firstunmanned aerial vehicle to reposition to the second location.
 7. Themethod of claim 1, wherein a distance between the first unmanned aerialvehicle and the first vehicle is the same when the first unmanned aerialvehicle is at the first location and when the first unmanned aerialvehicle is at the second location.
 8. The method of claim 1, wherein theat least one of the one or more second images shows a front region ofthe first vehicle or a rear region of the first vehicle.
 9. The methodof claim 1, wherein causing repositioning of the first unmanned aerialvehicle from the first location to the second location comprises:determining a dynamic characteristic of the first vehicle; determiningthat a magnitude of the dynamic characteristic exceeds a firstthreshold; and causing repositioning, in response to the determinationthat the magnitude of the dynamic characteristic exceeds the firstthreshold, of the first unmanned aerial vehicle from the first locationto the second location.
 10. The method of claim 9, wherein the dynamiccharacteristic is a ground speed of the first vehicle.
 11. The method ofclaim 1, wherein causing repositioning of the first unmanned aerialvehicle from the first location to the second location comprises:determining, based on at least one of the one or more first images, thatthe first vehicle has committed a parking violation; and causingrepositioning, in response to the determination that the first vehiclehas committed the parking violation, of the first unmanned aerialvehicle from the first location to the second location.
 12. The methodof claim 11, wherein determining that the first vehicle has committedthe parking violation comprises determining, based on at least one ofthe one or more first images, that the first vehicle is parked in arestricted parking zone.
 13. The method of claim 11, wherein determiningthat the first vehicle has committed the parking violation comprisesdetermining, based on at least one of the one or more first images, thatthe first vehicle has a missing or expired parking permit.
 14. Themethod of claim 1, wherein the unique identifier of the first vehicle isa license plate number, a one-dimensional bar code, a matrix bar code, aparking permit identification number, a disabled placard identificationnumber, or a vehicle registration tag associated with the first vehicle.15. The method of claim 1, wherein the one or more first images show thefirst vehicle and a second vehicle, the method further comprising:determining, based on at least one of the one or more first images, thata second unique identifier of the second vehicle is identifiable in atleast one of the one or more first images.
 16. The method of claim 1,further comprising: receiving, at a third time before the first time,one or more third images from a second camera, the one or more thirdimages having a third field of view and showing the first vehicle, andthe third field of view being different from and at least partiallyoverlapping with the first field of view; determining an overlap areathat defines the overlap between the one or more first images and theone or more third images; and determining a handoff zone within theoverlap area, wherein tracking of the first vehicle is passed from thesecond camera to the first camera after the first vehicle enters thehandoff zone.
 17. The method of claim 16, wherein the second camera islocated on a second unmanned aerial vehicle different from the firstunmanned aerial vehicle.
 18. The method of claim 1, wherein causingrepositioning of the first unmanned aerial vehicle from the firstlocation to the second location having the second field of viewcomprises: determining a first path to the second location from thefirst location; determining that an obstruction is present along thefirst path; and determining, in response to the determination that theobstruction is present along the first path, a second path to the secondlocation from the first location that does not overlap with theobstruction, the second path being different from the first path. 19.The method of claim 1, wherein: receiving the one or more first imagesfrom the first camera of the first unmanned aerial vehicle located atthe first location comprises receiving two or more first images from thefirst camera of the first unmanned aerial vehicle located at the firstlocation; determining the first characteristic of the first vehiclecomprises: determining, based on at least one of the two or more firstimages, a first static characteristic of the first vehicle, anddetermining, based on at least two of the two or more first images, afirst dynamic characteristic of the first vehicle; receiving the one ormore second images from the first camera of the first unmanned aerialvehicle located at the second location comprises receiving two or moresecond images from the first camera of the first unmanned aerial vehicleat the second location; determining the second characteristic of thefirst vehicle comprises: determining, based on at least one of the twoor more second images, a second static characteristic of the firstvehicle, and determining, based on at least two of the two or moresecond images, a second dynamic characteristic of the first vehicle;comparing the second characteristic of the first vehicle to the firstcharacteristic of the first vehicle comprises: comparing the secondstatic characteristic of the first vehicle to the first staticcharacteristic of the first vehicle, and comparing the second dynamiccharacteristic of the first vehicle to the first dynamic characteristicof the first vehicle; determining that the second characteristic of thefirst vehicle is approximately equal to the first characteristic of thefirst vehicle comprises: determining, in response to the comparison ofthe second static characteristic to the first static characteristic,that the second static characteristic is approximately equal to thefirst static characteristic, and determining, in response to thecomparison of the second dynamic characteristic to the first dynamiccharacteristic, that the second dynamic characteristic is approximatelyequal to the first dynamic characteristic; and determining that thefirst vehicle is present in the second field of view of the first cameracomprises determining, in response to the approximate equivalencebetween the second static characteristic and the first staticcharacteristic and the approximate equivalence between the seconddynamic characteristic and the first dynamic characteristic, that thefirst vehicle is present in the two or more second images, wherein: thefirst static characteristic and the second static characteristic arecharacteristics of the first vehicle that are constant over time, andthe first dynamic characteristic and the second dynamic characteristicare characteristics of the first vehicle that change over time.
 20. Themethod of claim 19, wherein: the first static characteristic and thesecond static characteristic are each one or more of a vehicle make, avehicle model, a vehicle year, a vehicle color, a vehicle outline, avehicle size, and a vehicle dimension of the first vehicle, and thefirst dynamic characteristic and the second dynamic characteristic areeach one or more of a lane identifier, a direction, a ground speed, andan acceleration of the first vehicle.
 21. The method of claim 1, whereinthe one or more first images show at least one destination location, themethod further comprising: determining, based on at least one of the oneor more first images, that the first vehicle is stopped within the atleast one destination location at the first time.
 22. The method ofclaim 21, further comprising: receiving, at a third time that is afterthe second time, one or more third images from the first camera of thefirst unmanned aerial vehicle, the one or more third images having athird field of view and showing the first vehicle and the at least onedestination location; determining, based on at least one of the one ormore third images, that the first vehicle has left the at least onedestination location at the third time; indicating that the firstvehicle began use of the at least one destination location at the firsttime; and indicating that the first vehicle completed use of the atleast one destination location at the third time.
 23. The method ofclaim 22, further comprising: determining a cost for the first vehicleto occupy the at least one destination location from the first time tothe third time; and charging the cost to an account associated with thefirst vehicle.
 24. The method of claim 22, further comprising:determining a cost for the first vehicle to occupy the at least onedestination location from the first time to the third time; andreceiving payment information for the cost from a third-party parkingpayment system.
 25. The method of claim 24, wherein the third-partyparking payment system is a third-party pay-by-phone parking paymentsystem, a third-party pay-by-space parking payment system, a third-partypay-and-display parking payment system, a third-party pay-by-platepayment system, or a third-party curbside parking meter payment system.26. The method of claim 22, further comprising: obtaining paymentinformation for the at least one destination location from a third-partyparking payment system, the payment information specifying a period oftime for which payment was received by the third-party parking paymentsystem for use of the at least one destination location by the firstvehicle; and determining that the use of the at least one destinationlocation by the first vehicle from the first time to the third time isoutside of the period of time received from the third-party parkingpayment system.
 27. The method of claim 26, further comprising:providing, in response to the determination that the use of the at leastone destination location by the first vehicle from the first time to thethird time is outside of the period of time received from thethird-party parking payment system, data comprising a unique identifierof the first vehicle, the period of time received from the third-partyparking payment system, an identifier of the destination location, thefirst time, and the third time.
 28. The method of claim 22, furthercomprising: obtaining a description of one or more restrictions for useof the at least one destination location; determining that the use ofthe at least one destination location by the first vehicle from thefirst time to the third time violates at least one of the one or morerestrictions; and indicating the at least one of the one or morerestrictions violated by the first vehicle.
 29. A method of tracking avehicle using an unmanned aerial vehicle, the method comprising:capturing, at a first time, one or more first images from a first cameraof a first unmanned aerial vehicle located at a first location, the oneor more first images having a first field of view and showing a firstvehicle; determining, based on at least one of the one or more firstimages, a first characteristic of the first vehicle; determining that aunique identifier of the first vehicle is not identifiable in at leastone of the one or more first images; repositioning, in response to thedetermination that the unique identifier of the first vehicle is notidentifiable in at least one of the one or more first images, the firstunmanned aerial vehicle from the first location to a second location,the second location being different from the first location; capturing,at a second time that is after the first time, one or more second imagesfrom the first camera of the first unmanned aerial vehicle, the one ormore second images having a second field of view and showing the firstvehicle, the second field of view being different from the first fieldof view; determining, based on at least one of the one or more secondimages, a second characteristic of the first vehicle; comparing thesecond characteristic of the first vehicle to the first characteristicof the first vehicle; determining, in response to the comparison of thesecond characteristic of the first vehicle to the first characteristicof the first vehicle, that the second characteristic of the firstvehicle is approximately equal to the first characteristic of the firstvehicle; determining, in response to the approximate equivalence of thesecond characteristic of the first vehicle and the first characteristicof the first vehicle, that the first vehicle is present in the one ormore second images; determining, based on at least one of the one ormore second images, the unique identifier of the first vehicle; andidentifying the unique identifier, wherein the steps of capturing,determining, repositioning, comparing and identifying are performed byone or more processors.
 30. A storage device storing a computer programfor tracking a vehicle using an unmanned aerial vehicle, the computerprogram comprising one or more code segments that, when executed, causeone or more processors to: receive, at a first time, one or more firstimages from a first camera of a first unmanned aerial vehicle located ata first location, the one or more first images having a first field ofview and showing a first vehicle; determine, based on at least one ofthe one or more first images, a first characteristic of the firstvehicle; determine that a unique identifier of the first vehicle is notidentifiable in at least one of the one or more first images; causerepositioning, in response to the determination that the uniqueidentifier of the first vehicle is not identifiable in the at least oneof the one or more first images, of the first unmanned aerial vehiclefrom the first location to a second location, the second location beingdifferent from the first location; receive, at a second time that isafter the first time, one or more second images from the first camera ofthe first unmanned aerial vehicle, the one or more second images havinga second field of view and showing the first vehicle, the second fieldof view being different from the first field of view; determine, basedon at least one of the one or more second images, a secondcharacteristic of the first vehicle; compare the second characteristicof the first vehicle to the first characteristic of the first vehicle;determine, in response to the comparison of the second characteristic ofthe first vehicle to the first characteristic of the first vehicle, thatthe second characteristic of the first vehicle is approximately equal tothe first characteristic of the first vehicle; determine, in response tothe approximate equivalence of the second characteristic of the firstvehicle and the first characteristic of the first vehicle, that thefirst vehicle is present in the one or more second images; determine,based on at least one of the one or more second images, the uniqueidentifier of the first vehicle; and identify the unique identifier.